-- Logs begin at Thu 2022-10-06 10:26:02 UTC, end at Thu 2022-10-06 11:29:48 UTC. -- Oct 06 10:49:32.281034 dv-cinder-860209-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 06 10:49:32.293373 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 6 10:49:32 2022] *** Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: nodename: dv-cinder-860209-2-iscsi Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: machine: x86_64 Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: clock source: unix Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: pcre jit disabled Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: detected number of CPU cores: 4 Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: current working directory: / Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: detected binary path: /usr/bin/uwsgi-core Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: your processes number limit is 47680 Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: your memory page size is 4096 bytes Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: detected max file descriptor number: 2048 Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: lock engine: pthread robust mutexes Oct 06 10:49:32.294405 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: thunder lock: enabled Oct 06 10:49:32.295999 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 06 10:49:32.295999 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 06 10:49:32.334371 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: Python main interpreter initialized at 0x55b1220819c0 Oct 06 10:49:32.334371 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: python threads support enabled Oct 06 10:49:32.334371 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: your server socket listen backlog is limited to 100 connections Oct 06 10:49:32.334371 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: your mercy for graceful operations on workers is 90 seconds Oct 06 10:49:32.334796 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: mapped 403077 bytes (393 KB) for 2 cores Oct 06 10:49:32.334953 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: *** Operational MODE: preforking *** Oct 06 10:49:32.335870 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: *** uWSGI is running in multiple interpreter mode *** Oct 06 10:49:32.335870 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: spawned uWSGI master process (pid: 81395) Oct 06 10:49:32.336088 dv-cinder-860209-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 06 10:49:32.339867 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 1 (pid: 81403, cores: 1) Oct 06 10:49:32.339867 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 2 (pid: 81404, cores: 1) Oct 06 10:49:32.339867 dv-cinder-860209-2-iscsi devstack@keystone.service[81395]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 06 10:49:36.594531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 10:49:36.594531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Oct 06 10:49:36.595385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 10:49:36.595385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: from cryptography.utils import int_from_bytes Oct 06 10:49:36.595806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 10:49:36.595806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Oct 06 10:49:36.595958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 10:49:36.595958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: from cryptography.utils import int_from_bytes Oct 06 10:49:36.644718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 06 10:49:36.644878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 06 10:49:36.644976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 06 10:49:36.645067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 06 10:49:36.645148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] command line args: [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 06 10:49:36.645221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] command line args: [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 06 10:49:36.645298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 06 10:49:36.645377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 06 10:49:36.645455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ================================================================================ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 06 10:49:36.645533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ================================================================================ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 06 10:49:36.645669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] admin_token = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.645751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.645912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config_dir = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.645999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_dir = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.646102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.646190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.646289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config_source = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.646375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_source = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.646458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] debug = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.646537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] debug = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.646648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_config_append = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.648960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_config_append = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_dir = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_dir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.649942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.650952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651501 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.651943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.652062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.652224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.652432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.652612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] publish_errors = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.652736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.652862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.652967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] publish_errors = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.653942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] standard_threads = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] standard_threads = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.654885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_journal = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_json = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_journal = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_stderr = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_json = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_syslog = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.655904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_stderr = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.656035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.656158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_syslog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.656280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.656385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 10:49:36.656503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.656618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.656745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.656867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.656982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.657096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.657210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.657323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.657704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.657828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.657943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.658995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.659118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.659233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.659347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.659461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.659575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.659689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.660944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.661957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.662072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.662195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.662311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.662425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.662539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.662653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.663290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.663290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.663694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.663818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.663951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.664994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.665917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.666957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.667980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.668924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.669038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.669151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.669276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.669408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.669615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.669867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.669987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.670127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.670283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.670407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.670577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.670717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.670834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.670949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.671952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672783 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.672900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.673966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674316 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.674902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.675907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.676992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.677975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.678096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.678212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.678327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.678497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.679982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.680097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.680212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.680328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.680527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.680655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.680772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.680895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.681963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.682915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.683937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.external = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.external = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.684927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.685050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.685177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.password = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.685302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.685420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.token = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.685756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.password = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.685873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.685996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.token = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.686952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.687925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.688985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.689980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.690941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.691921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.692998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.693939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.694968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.695918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.696985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.697928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.698987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.699967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.700917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.701982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702100 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.702910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.703740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.704897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.705960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.706926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.707886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.708949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.709903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.710938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.711897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.712979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.713916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.714988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.715976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.716924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.717932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.718993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.719222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.719338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.719455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.719571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.719688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.719688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.719934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.720987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.721922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.722851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.723991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.724942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.725058 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.725175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.725787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.725915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.726936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.727929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.728887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.729948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.730186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.730304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.730429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.730548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.730664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.730781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.730897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.731883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.732967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.733911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.734978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.735096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 06 10:49:36.735212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.735212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.735212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.735212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.735212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.735212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 10:49:36.735212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 06 10:49:36.760174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.760401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.760625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.760987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.761220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.761446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.768914 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.769782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.798145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.798371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.798719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.798953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.852231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.852706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.900691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.900864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.901504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:36.901743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.057757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.057979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.058866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.059003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.059218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.082984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.083117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.083198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.083325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.083413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.083535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.094421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.094772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.107644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.107757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.107895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.107995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.108981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.109062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 10:49:37.126223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.126413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.126498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 10:49:37.126626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 10:49:37.126759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.126959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 10:49:37.127048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.127158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 10:49:37.127238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.127431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.127512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.127692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.127773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.127974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.128056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.128230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.137672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.137900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.137984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.138178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.138261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.138447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.138537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.138730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.145414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.145738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.148367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.148648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.148785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.148927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.149071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.149207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.149357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.149490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.149639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.149749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.149923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.150003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.150186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.150268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.150457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.150576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.150771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.150868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.151000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.151081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.151273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.151540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.174433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.175185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.175903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.176188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.176685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.176971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.181644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.181925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.182204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.182285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.182489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.182767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.188505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.188791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.189133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.189422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.194882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.195171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.195455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.195683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.195766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.196897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.197099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.197370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.197638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.212891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.213167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.213935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.214214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.217020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.218109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.220252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.220547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.221375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.221674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.224761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.226046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.228161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.229284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.231379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.231676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.231990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.232270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.232555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.232634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.232838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.232929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.233184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.233475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.233756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.234046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.248612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.248896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.249163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.249425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.250091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.250382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.250651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.250914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.256245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.258356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.258450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.259929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.260065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.260224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.261462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.261739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.262880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.263163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.263662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.263752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.264127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.264534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.264820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.265103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.265384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.265783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.275971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.276262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.276543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.276825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.277205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.277494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.277772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.278702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.278994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.279320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.279953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.280245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.280525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.294079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.296740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.297332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.299653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.300255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.301737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.302597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.303207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.304268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.304658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.304743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.306137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.306418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.306818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.310996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.312487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.313220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.314683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.315361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.315752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.316058 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.316330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.317591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.317981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.318268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.318545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.324185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.324478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.324757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.326536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.326828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.327104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.331615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.331920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.332201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.332483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.332749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.333017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.333285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.333557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.333823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.334051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 10:49:37.334133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.334333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.334415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.334653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.334879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.335048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.335161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.335342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.335455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.335611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.335917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.336189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.336460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.336743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.337022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.337424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.337705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.337976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 10:49:37.370335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b1220819c0 pid: 81404 (default app) Oct 06 10:49:37.373346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b1220819c0 pid: 81403 (default app) Oct 06 10:49:37.373691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97df38-350d-4bd3-a593-181ef2f1a993 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:37.373922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97df38-350d-4bd3-a593-181ef2f1a993 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:37.374192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97df38-350d-4bd3-a593-181ef2f1a993 None None] PATH_INFO: `/v3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:37.378344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Thu Oct 6 10:49:37 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:44.172610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ca5f0-37fa-43c6-8f36-96947bd53102 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:44.173127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ca5f0-37fa-43c6-8f36-96947bd53102 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:44.173231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ca5f0-37fa-43c6-8f36-96947bd53102 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:44.177461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:49:44.183446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:44.184067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:44.184067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:44.254007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: INFO dbcounter [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] Registered counter for database keystone Oct 06 10:49:44.273056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG oslo_db.sqlalchemy.engines [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81404) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 06 10:49:44.279216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writer thread running {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 06 10:49:44.457633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG passlib.handlers.bcrypt [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81404) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 06 10:49:44.457941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG passlib.handlers.bcrypt [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81404) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 06 10:49:44.515902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:44.517538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:44.645838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 06 10:49:45.115748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ca5f0-37fa-43c6-8f36-96947bd53102 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:45.184777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: INFO dbcounter [None req-e23ca5f0-37fa-43c6-8f36-96947bd53102 None None] Registered counter for database keystone Oct 06 10:49:45.206056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG oslo_db.sqlalchemy.engines [None req-e23ca5f0-37fa-43c6-8f36-96947bd53102 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81403) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 06 10:49:45.212988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writer thread running {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 06 10:49:45.329692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ca5f0-37fa-43c6-8f36-96947bd53102 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:45.334514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7457adf-6f1a-4a60-92bc-90767ec330bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:45.335145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7457adf-6f1a-4a60-92bc-90767ec330bb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:45.335421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7457adf-6f1a-4a60-92bc-90767ec330bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:45.335692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7457adf-6f1a-4a60-92bc-90767ec330bb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:45.343954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7457adf-6f1a-4a60-92bc-90767ec330bb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:45.395184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f7457adf-6f1a-4a60-92bc-90767ec330bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 06 10:49:45.395533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7457adf-6f1a-4a60-92bc-90767ec330bb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:45.405971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Oct 6 10:49:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 06 10:49:45.412253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:45.429579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ea138-34bb-4340-981f-70c90e3d01a1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:45.433935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a2222-c573-4686-9e90-d8f049b44f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:45.435037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a2222-c573-4686-9e90-d8f049b44f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:45.435320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a2222-c573-4686-9e90-d8f049b44f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:45.435597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a2222-c573-4686-9e90-d8f049b44f16 admin admin] PATH_INFO: `/v3/projects/46752b92e78642e4b79f5ef50f2e51da` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:45.437214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a2222-c573-4686-9e90-d8f049b44f16 admin admin] RBAC: Authorizing `identity:get_project(project_id=46752b92e78642e4b79f5ef50f2e51da)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:45.492351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8c2a2222-c573-4686-9e90-d8f049b44f16 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 06 10:49:45.492708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a2222-c573-4686-9e90-d8f049b44f16 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:45.497294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Thu Oct 6 10:49:45 2022] GET /identity/v3/projects/46752b92e78642e4b79f5ef50f2e51da => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:47.638438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18868a-bf24-432c-a8aa-4c43a2d89897 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:47.638995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18868a-bf24-432c-a8aa-4c43a2d89897 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:47.638995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18868a-bf24-432c-a8aa-4c43a2d89897 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:47.643170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:47.649204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bce5a6-a4bb-40f2-854d-7f3c2610e149 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:47.649639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bce5a6-a4bb-40f2-854d-7f3c2610e149 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:47.649767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bce5a6-a4bb-40f2-854d-7f3c2610e149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:47.669953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-55bce5a6-a4bb-40f2-854d-7f3c2610e149 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:47.671467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-55bce5a6-a4bb-40f2-854d-7f3c2610e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:47.679380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 06 10:49:47.947731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18868a-bf24-432c-a8aa-4c43a2d89897 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:47.960548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18868a-bf24-432c-a8aa-4c43a2d89897 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:47.964714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0252c-8d2f-4980-8d4d-cea14c56052c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:47.965314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0252c-8d2f-4980-8d4d-cea14c56052c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:47.965613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0252c-8d2f-4980-8d4d-cea14c56052c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:47.965888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0252c-8d2f-4980-8d4d-cea14c56052c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:47.973521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0252c-8d2f-4980-8d4d-cea14c56052c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:47.975471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0252c-8d2f-4980-8d4d-cea14c56052c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:47.984104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Oct 6 10:49:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 10:49:47.989406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bce5a6-a4bb-40f2-854d-7f3c2610e149 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:47.999171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bce5a6-a4bb-40f2-854d-7f3c2610e149 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:48.003487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df495c-860e-435e-827d-4b1b2bb9fbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:48.004150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df495c-860e-435e-827d-4b1b2bb9fbf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:48.004434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df495c-860e-435e-827d-4b1b2bb9fbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:48.004710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df495c-860e-435e-827d-4b1b2bb9fbf4 admin admin] PATH_INFO: `/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:48.007161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df495c-860e-435e-827d-4b1b2bb9fbf4 admin admin] RBAC: Authorizing `identity:get_user(user_id=dd261f057dda47fbbc28e2f5e9b2f3e7)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:48.009020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df495c-860e-435e-827d-4b1b2bb9fbf4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:48.010548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Thu Oct 6 10:49:47 2022] GET /identity/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:50.220304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c029-92d5-4762-bd86-c5ea29a14f07 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:50.220846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c029-92d5-4762-bd86-c5ea29a14f07 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:50.220846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c029-92d5-4762-bd86-c5ea29a14f07 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:50.222189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:50.227979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3f7f-7af2-437f-a577-7a4194f50040 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:50.228410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3f7f-7af2-437f-a577-7a4194f50040 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:50.228666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3f7f-7af2-437f-a577-7a4194f50040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:50.238077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd4fb3-79eb-4a30-86fc-7499f3e94986 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:50.238361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd4fb3-79eb-4a30-86fc-7499f3e94986 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:50.238636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd4fb3-79eb-4a30-86fc-7499f3e94986 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:50.240677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:49:50.245760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:50.246041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:50.246312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:50.247299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8f1c3f7f-7af2-437f-a577-7a4194f50040 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:50.248518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8f1c3f7f-7af2-437f-a577-7a4194f50040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:50.288715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81403) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 06 10:49:50.289105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81403) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 06 10:49:50.330635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:50.337906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:50.339492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:50.440261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:49:50.601361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c3f7f-7af2-437f-a577-7a4194f50040 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:50.626049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:50.707095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c3f7f-7af2-437f-a577-7a4194f50040 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:50.707303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:50.741843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:50.742476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:50.742762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:50.743040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:50.754254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:49:50.763785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:49:50.764744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:49:50.765598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:49:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:49:50.770663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:50.810815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9deb55-7496-4db8-86b4-27ecd2dbb70d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:50.813110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:50.849301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:50.850167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:50.850449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:50.850721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:50.854186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:50.872967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a8b28-efe8-4598-b6cb-690f254b5d62 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:50.908096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:50.908695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:50.909228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:50.909501 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:50.910165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:49:50.910464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:50.911916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:50.919328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:50.920882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:50.924149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:49:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:50.930043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:51.019644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:49:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:49:51.025494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:51.028184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:51.032691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf172d5a-5c96-4246-b989-b59bd42177a3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:51.066947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:51.067577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:51.068477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:51.068758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] PATH_INFO: `/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:51.070018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] RBAC: Authorizing `identity:get_user(user_id=dd261f057dda47fbbc28e2f5e9b2f3e7)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:51.071473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:51.073246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:49:50 2022] GET /identity/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:51.078396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:51.093141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:51.123317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f526ad-7de4-483f-9be0-9e2bac81a4c7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:51.157012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79431a8-1584-470a-b536-e56954794007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:51.157871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79431a8-1584-470a-b536-e56954794007 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:51.158150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79431a8-1584-470a-b536-e56954794007 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:51.158425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79431a8-1584-470a-b536-e56954794007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:51.177792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79431a8-1584-470a-b536-e56954794007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:51.178400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92a74d-660a-42f1-89ef-ae35cacef8af None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:51.179586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79431a8-1584-470a-b536-e56954794007 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e79431a8-1584-470a-b536-e56954794007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:49:51.198503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:49:51.199429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:49:51.200447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:49:51.200447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:49:51.200447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:49:51.200447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:49:51.205431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79431a8-1584-470a-b536-e56954794007 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:51.212158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:51.212794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:51.213077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:51.213353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:51.214477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:51.216024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:51.217714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:51.223202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:51.261131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:51.306022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:51.306937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79431a8-1584-470a-b536-e56954794007 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:51.325878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e47e8-bd18-4717-93fd-340f49dbedd7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:51.340941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675d749-556e-43af-ba6c-03935be39413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:51.341562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675d749-556e-43af-ba6c-03935be39413 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:51.341847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675d749-556e-43af-ba6c-03935be39413 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:51.342129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675d749-556e-43af-ba6c-03935be39413 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:51.342774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675d749-556e-43af-ba6c-03935be39413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:49:51.343084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675d749-556e-43af-ba6c-03935be39413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:51.344506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675d749-556e-43af-ba6c-03935be39413 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:51.359465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83de01c-55a6-4cc3-a592-28ba146efca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:51.360091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de01c-55a6-4cc3-a592-28ba146efca5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:51.360374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de01c-55a6-4cc3-a592-28ba146efca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:51.360646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de01c-55a6-4cc3-a592-28ba146efca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:51.361375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83de01c-55a6-4cc3-a592-28ba146efca5 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.domain.id=default, user.id=dd261f057dda47fbbc28e2f5e9b2f3e7) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:49:51.361678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83de01c-55a6-4cc3-a592-28ba146efca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:51.361944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:51.363257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83de01c-55a6-4cc3-a592-28ba146efca5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:51.371264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Oct 6 10:49:51 2022] GET /identity/v3/role_assignments?user.id=dd261f057dda47fbbc28e2f5e9b2f3e7&scope.domain.id=default&role.id=4c41d809cc274ff5871dc8092f05715a => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:53.739151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844e0b1-dc52-4fb6-9ba1-add831723170 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:53.740681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844e0b1-dc52-4fb6-9ba1-add831723170 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:53.741166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844e0b1-dc52-4fb6-9ba1-add831723170 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:53.742976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:53.748785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf8ac3-6879-4fa6-a394-0b823e4e26be None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:53.749649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf8ac3-6879-4fa6-a394-0b823e4e26be None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:53.750213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf8ac3-6879-4fa6-a394-0b823e4e26be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:53.769625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f3cf8ac3-6879-4fa6-a394-0b823e4e26be None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:53.771701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f3cf8ac3-6879-4fa6-a394-0b823e4e26be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:53.841577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 12/23] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:54.234884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c9101-9eec-4620-92a6-130730a2d9cc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.235556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c9101-9eec-4620-92a6-130730a2d9cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.236110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c9101-9eec-4620-92a6-130730a2d9cc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.237714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:49:54.243309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0d43-8968-4d08-b1f1-faf526414b38 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.243862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0d43-8968-4d08-b1f1-faf526414b38 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.244360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0d43-8968-4d08-b1f1-faf526414b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.270466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cf8a0d43-8968-4d08-b1f1-faf526414b38 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:54.272609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf8a0d43-8968-4d08-b1f1-faf526414b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:54.340348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c9101-9eec-4620-92a6-130730a2d9cc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:54.358518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 13/25] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:54.388906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce593422-602f-4c5d-9664-d753ae75b5f1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.389202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce593422-602f-4c5d-9664-d753ae75b5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.389464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce593422-602f-4c5d-9664-d753ae75b5f1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.391130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 14/26] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:54.399064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779eab-f105-45ac-a215-66770ca1b456 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.399351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779eab-f105-45ac-a215-66770ca1b456 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.399629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779eab-f105-45ac-a215-66770ca1b456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.426966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:54.444851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-71779eab-f105-45ac-a215-66770ca1b456 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:54.446047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-71779eab-f105-45ac-a215-66770ca1b456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:54.571658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c9101-9eec-4620-92a6-130730a2d9cc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:54.605492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:54.616456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 15/27] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:54.636422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:54.637066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.637347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.637624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.657111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:54.658554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:49:54.680530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:49:54.681413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:49:54.682263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:49:54.682263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:49:54.682263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:49:54.682263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 13/28] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:49:54.687738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779eab-f105-45ac-a215-66770ca1b456 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:54.723029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:54.884642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779eab-f105-45ac-a215-66770ca1b456 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:54.890362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe7747-cb1b-4a97-8ce4-01a6c5aa9ad5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:54.957050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b84fd-1e46-478a-94b7-2abaa4e8d9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:54.957676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b84fd-1e46-478a-94b7-2abaa4e8d9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.957960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b84fd-1e46-478a-94b7-2abaa4e8d9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.958478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b84fd-1e46-478a-94b7-2abaa4e8d9d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.959069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b84fd-1e46-478a-94b7-2abaa4e8d9d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:49:54.959376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b84fd-1e46-478a-94b7-2abaa4e8d9d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:54.960810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b84fd-1e46-478a-94b7-2abaa4e8d9d3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:54.962956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7793fdb-7061-498e-8d48-446136a106f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:54.963594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7793fdb-7061-498e-8d48-446136a106f2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.963894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7793fdb-7061-498e-8d48-446136a106f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.964173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7793fdb-7061-498e-8d48-446136a106f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.967855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:54.993093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 16/29] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:54.994367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b767c1-5862-42b4-bda4-69898b710b3f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:54.994656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b767c1-5862-42b4-bda4-69898b710b3f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:54.994936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b767c1-5862-42b4-bda4-69898b710b3f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:54.996453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 17/30] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:54.998157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7793fdb-7061-498e-8d48-446136a106f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:49:55.001156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b767c1-5862-42b4-bda4-69898b710b3f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e7793fdb-7061-498e-8d48-446136a106f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:49:55.008881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:49:55.009778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:49:55.010535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:49:55.011136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 14/31] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:49:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:49:55.011806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55eff2-2365-4598-b175-d6b100747a67 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.012222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55eff2-2365-4598-b175-d6b100747a67 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.012500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55eff2-2365-4598-b175-d6b100747a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.032817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-df55eff2-2365-4598-b175-d6b100747a67 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:55.034359 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-df55eff2-2365-4598-b175-d6b100747a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:55.060347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:55.111924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 15/32] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:55.111924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55eff2-2365-4598-b175-d6b100747a67 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:55.134790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b767c1-5862-42b4-bda4-69898b710b3f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:55.202051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:55.202714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.202994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.203261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.204024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:55.205408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:55.268757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55eff2-2365-4598-b175-d6b100747a67 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:55.355815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 18/33] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Oct 6 10:49:55 2022] POST /identity/v3/roles => generated 219 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:49:55.356327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:55.363384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:55.364171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.365330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.365613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.366547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:55.368078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:55.461514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:55.534503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 16/34] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:49:55 2022] POST /identity/v3/projects => generated 301 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:49:55.535681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:55.553760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:55.562523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5417d-8072-4c60-8b82-6436209d5a89 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:55.619267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779df9e3-e2cc-4051-8fc5-8de81913b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:55.620145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779df9e3-e2cc-4051-8fc5-8de81913b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.622812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779df9e3-e2cc-4051-8fc5-8de81913b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.623187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779df9e3-e2cc-4051-8fc5-8de81913b8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.625509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779df9e3-e2cc-4051-8fc5-8de81913b8f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:49:55.625823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779df9e3-e2cc-4051-8fc5-8de81913b8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:55.628476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779df9e3-e2cc-4051-8fc5-8de81913b8f6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:55.651215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 19/35] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:55.653807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471986a-b2f1-4fb0-9db1-68c808eba102 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.654095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471986a-b2f1-4fb0-9db1-68c808eba102 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.654372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471986a-b2f1-4fb0-9db1-68c808eba102 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.656851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 20/36] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:55.659353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471986a-b2f1-4fb0-9db1-68c808eba102 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:55.712703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533fbcd-a0df-4a89-b7d6-e747d0f692d4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:55.757752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35b404-ee5a-407b-ae39-3e7370929a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:55.758411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35b404-ee5a-407b-ae39-3e7370929a2c None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.758701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35b404-ee5a-407b-ae39-3e7370929a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.758984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35b404-ee5a-407b-ae39-3e7370929a2c None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.759701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35b404-ee5a-407b-ae39-3e7370929a2c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:55.763549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35b404-ee5a-407b-ae39-3e7370929a2c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:55.786284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 17/37] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Oct 6 10:49:55 2022] POST /identity/v3/roles => generated 225 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:49:55.787981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449335-e5ac-48ab-a181-ce7303aa26b1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.788280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449335-e5ac-48ab-a181-ce7303aa26b1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.788613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449335-e5ac-48ab-a181-ce7303aa26b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.812956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cf449335-e5ac-48ab-a181-ce7303aa26b1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:55.814702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf449335-e5ac-48ab-a181-ce7303aa26b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:55.895870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471986a-b2f1-4fb0-9db1-68c808eba102 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:55.912724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:55.915822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:55.918835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 18/38] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:55.920463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1313-a21e-4442-a4e9-9293a331c1a7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.920908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1313-a21e-4442-a4e9-9293a331c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.921889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1313-a21e-4442-a4e9-9293a331c1a7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.923787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 19/39] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:55.930810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707bcff-9ce5-4717-81af-1a9482d25f47 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.931703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707bcff-9ce5-4717-81af-1a9482d25f47 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.932696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707bcff-9ce5-4717-81af-1a9482d25f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.949473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:55.950144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:55.950428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:55.950712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] PATH_INFO: `/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:55.952112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] RBAC: Authorizing `identity:get_user(user_id=dd261f057dda47fbbc28e2f5e9b2f3e7)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:55.953624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:55.960482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 21/40] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:49:55 2022] GET /identity/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7 => generated 247 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:55.961679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:55.964933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5707bcff-9ce5-4717-81af-1a9482d25f47 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:55.966957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5707bcff-9ce5-4717-81af-1a9482d25f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:56.054113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 20/41] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:56.084396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eca48-113f-4410-93da-5c741fccd9cd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:56.118778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:56.119405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:56.119687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:56.119994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:56.121212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:56.122656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:56.124253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 22/42] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:49:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:56.130605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707bcff-9ce5-4717-81af-1a9482d25f47 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:56.192290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:56.234450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707bcff-9ce5-4717-81af-1a9482d25f47 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:56.250982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:56.270369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:56.276784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:56.277456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:56.277746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:56.278027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] PATH_INFO: `/v3/domains/default/users/dd261f057dda47fbbc28e2f5e9b2f3e7/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:56.280215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=dd261f057dda47fbbc28e2f5e9b2f3e7, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:56.281671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:56.299166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 21/43] 10.222.0.52 () {58 vars in 1361 bytes} [Thu Oct 6 10:49:56 2022] PUT /identity/v3/domains/default/users/dd261f057dda47fbbc28e2f5e9b2f3e7/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:49:56.305852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0d11a-4246-452a-b399-1f762f9fea97 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:56.325689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:56.351601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:56.352550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:56.352836 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:56.353111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:56.354008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:56.356659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:56.374697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 23/44] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Oct 6 10:49:56 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:49:56.457810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2180f-1c39-459c-b34f-77b3e8e06608 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:56.498834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9833b3-3855-450f-9a16-9b24e231b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:56.499505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9833b3-3855-450f-9a16-9b24e231b267 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:56.499793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9833b3-3855-450f-9a16-9b24e231b267 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:56.500100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9833b3-3855-450f-9a16-9b24e231b267 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:56.501318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9833b3-3855-450f-9a16-9b24e231b267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:56.502808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9833b3-3855-450f-9a16-9b24e231b267 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:56.504504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 22/45] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:56.511165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:56.622050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39678ba7-ed4e-4be5-b43c-81ffd5b436be None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:56.636375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:56.660294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ec019-ff4d-4a8e-aa24-acdee83aa268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:56.660910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ec019-ff4d-4a8e-aa24-acdee83aa268 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:56.661195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ec019-ff4d-4a8e-aa24-acdee83aa268 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:56.662042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ec019-ff4d-4a8e-aa24-acdee83aa268 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:56.662710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ec019-ff4d-4a8e-aa24-acdee83aa268 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:56.664193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ec019-ff4d-4a8e-aa24-acdee83aa268 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:56.693568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 24/46] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:49:56 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:49:58.494565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ec1c5-1de7-4fb8-918e-7298a1a794f4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:58.495106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ec1c5-1de7-4fb8-918e-7298a1a794f4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:58.495195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ec1c5-1de7-4fb8-918e-7298a1a794f4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:58.496576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 23/47] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:58.501946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb40df-2949-4daa-b0b6-040d7e9e2105 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:58.502355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb40df-2949-4daa-b0b6-040d7e9e2105 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:58.502625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb40df-2949-4daa-b0b6-040d7e9e2105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:58.521627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fddb40df-2949-4daa-b0b6-040d7e9e2105 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:58.523137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fddb40df-2949-4daa-b0b6-040d7e9e2105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:58.588643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 25/48] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:58.850585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853088d-a554-4b67-9bc8-99d9a618ee53 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:58.850867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853088d-a554-4b67-9bc8-99d9a618ee53 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:58.851146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853088d-a554-4b67-9bc8-99d9a618ee53 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:58.852659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 24/49] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:49:58.857878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fd1a7-f596-4112-ade5-f133e5801608 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:58.858172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fd1a7-f596-4112-ade5-f133e5801608 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:58.858328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853088d-a554-4b67-9bc8-99d9a618ee53 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:58.858441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fd1a7-f596-4112-ade5-f133e5801608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:58.876848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-513fd1a7-f596-4112-ade5-f133e5801608 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:49:58.878042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-513fd1a7-f596-4112-ade5-f133e5801608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:49:58.918293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:58.945842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 26/50] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:49:58.962324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853088d-a554-4b67-9bc8-99d9a618ee53 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:58.996640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:58.997277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:58.997561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:58.997838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:59.008170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:49:59.013424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:49:59.014397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:49:59.015273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 25/51] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:49:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:49:59.020353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fd1a7-f596-4112-ade5-f133e5801608 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:59.045721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:59.120785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fd1a7-f596-4112-ade5-f133e5801608 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:59.153297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:59.153904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:59.154187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:59.154465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:59.155128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:49:59.155434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:59.156866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:59.165489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 27/52] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:49:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:59.171133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:59.213299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:59.271819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552bcb-8e1e-483a-b3ca-bd55e3a91409 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:59.289293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:59.300402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:59.306468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:59.307107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:59.307392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:59.307665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] PATH_INFO: `/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:59.308830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] RBAC: Authorizing `identity:get_user(user_id=dd261f057dda47fbbc28e2f5e9b2f3e7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:59.310287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:59.311880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 26/53] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:49:59 2022] GET /identity/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:49:59.316688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ca82f-5fa3-402b-8ff8-045d765a692d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:59.317275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:59.351420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:59.352080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:59.352362 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:59.352636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:59.353763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:59.355219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:59.356821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 28/54] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:59.362254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:59.427274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce8ac-e6c5-4163-937e-924a33eb86d1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:59.460430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:59.461060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:59.461350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:59.461622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:59.462731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:59.464204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:59.465953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 27/55] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:49:59.469613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d63f8-2af3-4b2e-a681-671a4484d25b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:59.471428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:49:59.503203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bee64-ebc1-42e1-91ac-289be6d503f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:59.503803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bee64-ebc1-42e1-91ac-289be6d503f8 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:59.504111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bee64-ebc1-42e1-91ac-289be6d503f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:59.504384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bee64-ebc1-42e1-91ac-289be6d503f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:59.505655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bee64-ebc1-42e1-91ac-289be6d503f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:59.507102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bee64-ebc1-42e1-91ac-289be6d503f8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:59.526221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:49:59.535764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 29/56] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Oct 6 10:49:59 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:49:59.576702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276f72-71dd-4528-83db-b83402fb22e7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:49:59.612840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ffc71-5b9c-4f8c-962b-a7441fb3da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:49:59.613487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ffc71-5b9c-4f8c-962b-a7441fb3da38 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:49:59.613753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ffc71-5b9c-4f8c-962b-a7441fb3da38 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:49:59.614144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ffc71-5b9c-4f8c-962b-a7441fb3da38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:49:59.615130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ffc71-5b9c-4f8c-962b-a7441fb3da38 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.domain.id=default, user.id=dd261f057dda47fbbc28e2f5e9b2f3e7) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:49:59.615441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ffc71-5b9c-4f8c-962b-a7441fb3da38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:49:59.616961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ffc71-5b9c-4f8c-962b-a7441fb3da38 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:49:59.625648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 28/57] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Oct 6 10:49:59 2022] GET /identity/v3/role_assignments?user.id=dd261f057dda47fbbc28e2f5e9b2f3e7&scope.domain.id=default&role.id=4c41d809cc274ff5871dc8092f05715a => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:01.705016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d2a2-47b3-4562-93e3-78095b8c8786 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:01.705517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d2a2-47b3-4562-93e3-78095b8c8786 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:01.705517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d2a2-47b3-4562-93e3-78095b8c8786 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:01.706818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 30/58] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:01.712471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828a8d8-059d-4461-9df7-38489ce40978 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:01.712897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828a8d8-059d-4461-9df7-38489ce40978 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:01.713039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828a8d8-059d-4461-9df7-38489ce40978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:01.731751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0828a8d8-059d-4461-9df7-38489ce40978 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:01.732973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0828a8d8-059d-4461-9df7-38489ce40978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:01.900266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:02.169514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d2a2-47b3-4562-93e3-78095b8c8786 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:02.179073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:02.270808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d2a2-47b3-4562-93e3-78095b8c8786 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:02.304156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd625ee-39ba-48c5-a000-2f13648160de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:02.304777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd625ee-39ba-48c5-a000-2f13648160de None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:02.305059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd625ee-39ba-48c5-a000-2f13648160de None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:02.305332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd625ee-39ba-48c5-a000-2f13648160de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:02.306448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd625ee-39ba-48c5-a000-2f13648160de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:02.307913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd625ee-39ba-48c5-a000-2f13648160de None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:02.309459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 31/60] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:02.314840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828a8d8-059d-4461-9df7-38489ce40978 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:02.362496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:02.413576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828a8d8-059d-4461-9df7-38489ce40978 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:02.445581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973c2e8-e198-4620-a194-83fa9b2d67d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:02.446125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c2e8-e198-4620-a194-83fa9b2d67d9 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:02.446412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c2e8-e198-4620-a194-83fa9b2d67d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:02.446693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c2e8-e198-4620-a194-83fa9b2d67d9 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:02.447354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973c2e8-e198-4620-a194-83fa9b2d67d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:02.448837 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973c2e8-e198-4620-a194-83fa9b2d67d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:02.475326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 30/61] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:50:02 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:05.628006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987146b-43f4-41cc-8921-579e568b81b7 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:05.628924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987146b-43f4-41cc-8921-579e568b81b7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:05.630010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987146b-43f4-41cc-8921-579e568b81b7 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:05.631933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 32/62] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:05.636911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcba67-fbbf-47d4-a5e7-7d33946691d6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:05.637670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcba67-fbbf-47d4-a5e7-7d33946691d6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:05.638176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcba67-fbbf-47d4-a5e7-7d33946691d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:05.664913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ffdcba67-fbbf-47d4-a5e7-7d33946691d6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:05.666444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ffdcba67-fbbf-47d4-a5e7-7d33946691d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:05.754981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 31/63] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:06.073041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bb152-4a0c-4c50-b6b0-ae627099ff75 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.073612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bb152-4a0c-4c50-b6b0-ae627099ff75 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.074061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bb152-4a0c-4c50-b6b0-ae627099ff75 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.075588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 33/64] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:06.093464 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667123b7-496a-4e19-9afa-1477828431fe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.094042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667123b7-496a-4e19-9afa-1477828431fe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.094517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667123b7-496a-4e19-9afa-1477828431fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.120875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-667123b7-496a-4e19-9afa-1477828431fe None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:06.122450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-667123b7-496a-4e19-9afa-1477828431fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:06.154921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:06.203355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 32/65] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:06.203827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9527b3c-e3ad-4850-933f-7817e0e41882 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.204360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9527b3c-e3ad-4850-933f-7817e0e41882 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.204835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9527b3c-e3ad-4850-933f-7817e0e41882 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.206374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 34/66] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:06.211694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad76b6-3c08-4cd0-8079-b77aba1fd98c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.212010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad76b6-3c08-4cd0-8079-b77aba1fd98c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.212287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad76b6-3c08-4cd0-8079-b77aba1fd98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.240852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-57ad76b6-3c08-4cd0-8079-b77aba1fd98c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:06.242091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-57ad76b6-3c08-4cd0-8079-b77aba1fd98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:06.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9527b3c-e3ad-4850-933f-7817e0e41882 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:06.326541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 33/67] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:06.460386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:06.495644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9527b3c-e3ad-4850-933f-7817e0e41882 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:06.509933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad76b6-3c08-4cd0-8079-b77aba1fd98c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:06.547454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99322f80-54c5-4b02-a344-19dbb099cf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:06.548523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99322f80-54c5-4b02-a344-19dbb099cf72 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.549003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99322f80-54c5-4b02-a344-19dbb099cf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.549467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99322f80-54c5-4b02-a344-19dbb099cf72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.561252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99322f80-54c5-4b02-a344-19dbb099cf72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-99322f80-54c5-4b02-a344-19dbb099cf72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:06.567853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:06.568703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:06.569507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:06.572009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 35/68] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:06.573564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae5157-f7a8-47a6-a2d1-08efcb7e4c12 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.574110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae5157-f7a8-47a6-a2d1-08efcb7e4c12 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.574743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae5157-f7a8-47a6-a2d1-08efcb7e4c12 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.580149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 36/69] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:06.580149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275612-65eb-40bf-8327-3051cec051ce None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.580149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275612-65eb-40bf-8327-3051cec051ce None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.580149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275612-65eb-40bf-8327-3051cec051ce None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.580149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 37/70] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:06.580654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275612-65eb-40bf-8327-3051cec051ce None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:06.617129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:06.624617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad76b6-3c08-4cd0-8079-b77aba1fd98c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:06.659496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b029bad-bbaa-45b0-9f57-0ee9cda9e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:06.660962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029bad-bbaa-45b0-9f57-0ee9cda9e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.661247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029bad-bbaa-45b0-9f57-0ee9cda9e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.661529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029bad-bbaa-45b0-9f57-0ee9cda9e4c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.671865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b029bad-bbaa-45b0-9f57-0ee9cda9e4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1b029bad-bbaa-45b0-9f57-0ee9cda9e4c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:06.677940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:06.678838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:06.679603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 34/71] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:06.681487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744485-6653-48e2-99c5-2a34515cb65f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.681775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744485-6653-48e2-99c5-2a34515cb65f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.682051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744485-6653-48e2-99c5-2a34515cb65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.698900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-26744485-6653-48e2-99c5-2a34515cb65f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:06.699334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275612-65eb-40bf-8327-3051cec051ce None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:06.700120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-26744485-6653-48e2-99c5-2a34515cb65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:06.733338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707d214-7aa1-4978-96e9-fddde5a6b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:06.733949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d214-7aa1-4978-96e9-fddde5a6b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.734236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d214-7aa1-4978-96e9-fddde5a6b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.734514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d214-7aa1-4978-96e9-fddde5a6b7e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.735177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707d214-7aa1-4978-96e9-fddde5a6b7e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:06.735480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707d214-7aa1-4978-96e9-fddde5a6b7e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:06.736999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707d214-7aa1-4978-96e9-fddde5a6b7e1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:06.738359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:06.746286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 38/72] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:06.752169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ca7a6-2376-404a-9927-1bf3feab064a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.752464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ca7a6-2376-404a-9927-1bf3feab064a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.752753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ca7a6-2376-404a-9927-1bf3feab064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.765726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 35/73] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:06.766641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744485-6653-48e2-99c5-2a34515cb65f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:06.771318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bb6ca7a6-2376-404a-9927-1bf3feab064a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:06.772603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bb6ca7a6-2376-404a-9927-1bf3feab064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:06.839308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 39/74] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:06.840099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ca7a6-2376-404a-9927-1bf3feab064a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:06.865976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:06.886239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744485-6653-48e2-99c5-2a34515cb65f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:06.928411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:06.929053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:06.929341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:06.929617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:06.941543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:06.947714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:06.948567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:06.949730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 36/75] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:06.952650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:06.975268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ca7a6-2376-404a-9927-1bf3feab064a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.005904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.017331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.018314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.018749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.019211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.020807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:07.021325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.022924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.032312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 40/76] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:07.033525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.065435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d85d3-35cf-4375-b879-d1a4c5b59a2f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.101654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.102215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.102495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.102768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.118675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.120161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.121795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 37/77] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:06 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:07.122559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.125097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.142189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005e39-07d3-4bbe-9572-f0b52d2cd819 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.176321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.176985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.177264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.177540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.178225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:07.178526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.179911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.187682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 41/78] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:07.188414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.225542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbd64b-1cfc-483e-b4f1-a69f2d6c0656 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.240898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.258698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.259343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.259629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.259941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] PATH_INFO: `/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.261089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] RBAC: Authorizing `identity:get_user(user_id=dd261f057dda47fbbc28e2f5e9b2f3e7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.262550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.264458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 38/79] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:07.265235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.295599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94574bcf-234c-4171-b769-ab645ce0927e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.329290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.329922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.330213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.330496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.334849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.341330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:07.346633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:07.347497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 42/80] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:07.349125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.371003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925e474-b8de-42c8-b84f-6ccfa13cfa2a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.406529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.407287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.407585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.408148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.409288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.410803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.412419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 39/81] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:07.413193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.448676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d0709-c9c0-4cad-98ab-74cc72ee10d1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.459352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.483209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.483821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.484141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.484420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.514836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e67af-ec22-465e-935b-e21b5e2076f2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.550218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.550382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.551363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.551859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.552297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.552401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.553478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.554922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.556534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 40/82] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:07.557313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.570181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.649668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 43/83] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:07.650543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.662523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.664980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29604aa9-5def-41ad-96bc-970fc0f1e128 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.698713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.699315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.699595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.699893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.753195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128656b-4569-474f-952e-b66d0325f8b0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.768981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.770905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.778871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.788749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.789358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.789644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.789918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.790577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:07.790879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.792294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.800543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 44/84] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:07.802114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.871766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:07.875689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 41/85] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:07.880547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.902961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d30e6-79bd-439c-8b7b-24f71dabb9cf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.936801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:07.937404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:07.937720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:07.937952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:07.938600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:07.940080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:07.966188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 45/86] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:50:07 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:07.967013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:07.978555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9e8f-3064-4339-be4a-346cbeb70c07 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:07.987262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:08.021274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.021908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.022191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.022463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.094660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f17d0c-1fb5-4c3e-aa3c-e6a2a89b283d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.100557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.102062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.127039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:08.138431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.139562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.139938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.140841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.141465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.142941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.144671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 46/87] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:08.150217 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:08.222455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 42/88] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:08.223581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:08.234032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:08.278388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b9917-ced2-44cb-a8de-d953459c5130 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.320938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.321574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.321855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.322128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.323259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.324790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.326543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 47/89] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:08 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:08.327147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:08.343262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1870fed-3324-47af-82a4-fd6a6d5abf5b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.385014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.385655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.385935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.386208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.390040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.391522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.393241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 43/90] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:08 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:08.396672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:08.424608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:08.478163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb10494-e241-4c81-a0f8-5d88217121a4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.482294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:08.509284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9368a5-6189-4b39-bf2e-260005686be4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.523974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.524592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.524874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.525149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.526130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=7db7540927364ffabd7480f25538841b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:08.526443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.528673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.537163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 48/91] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:08 2022] GET /identity/v3/role_assignments?user.id=7db7540927364ffabd7480f25538841b&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:08.538333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:08.550608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.551879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.552344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.552793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.554148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.555864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.557729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 44/92] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:08 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:08.558673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:08.689512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256ccc-b2ea-4e91-b01a-2bd14a9b7ced None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.719645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3883d2-54ca-4231-8d45-6349ae2b1d07 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.759037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.759584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.759990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.760531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.761282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=dd261f057dda47fbbc28e2f5e9b2f3e7) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:08.761595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.763197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.766480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:08.770110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 45/93] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:08 2022] GET /identity/v3/role_assignments?user.id=dd261f057dda47fbbc28e2f5e9b2f3e7&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:08.771198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:08.781225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:08.783635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.784608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.785035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.786167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.888390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.890386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.909062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce3ee8-a14a-44ce-90a8-54f073c17341 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:08.956618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:08.957308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:08.957585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:08.957863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:08.958725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] RBAC: Adding query filter params (role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=7db7540927364ffabd7480f25538841b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:08.959037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:08.960568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:08.972572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 46/94] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:08 2022] GET /identity/v3/role_assignments?user.id=7db7540927364ffabd7480f25538841b&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=2372f2ee163c430ca4b943c833677192 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:09.052800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 49/95] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:09.058798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:09.147703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:09.193070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11ec6d-0146-4dd0-9510-8c97db2d035c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:09.246805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd465b4-6976-4233-af65-8f9a44c4375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:09.247479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd465b4-6976-4233-af65-8f9a44c4375a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:09.247827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd465b4-6976-4233-af65-8f9a44c4375a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:09.248135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd465b4-6976-4233-af65-8f9a44c4375a None admin] PATH_INFO: `/v3/projects/e8ad380e71d24628bd90f6888c32b821` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:09.249521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd465b4-6976-4233-af65-8f9a44c4375a None admin] RBAC: Authorizing `identity:get_project(project_id=e8ad380e71d24628bd90f6888c32b821)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:09.251122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd465b4-6976-4233-af65-8f9a44c4375a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:09.253149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 47/96] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:09 2022] GET /identity/v3/projects/e8ad380e71d24628bd90f6888c32b821 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:09.259524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:09.296314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:09.397507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e35ab-b1e2-47cb-a22b-323397956d4d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:09.446709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869b9d3-9f83-412b-be28-95cb7ae2b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:09.447323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869b9d3-9f83-412b-be28-95cb7ae2b297 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:09.447608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869b9d3-9f83-412b-be28-95cb7ae2b297 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:09.447957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869b9d3-9f83-412b-be28-95cb7ae2b297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:09.448683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869b9d3-9f83-412b-be28-95cb7ae2b297 None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=e8ad380e71d24628bd90f6888c32b821, user.id=7db7540927364ffabd7480f25538841b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:09.448989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869b9d3-9f83-412b-be28-95cb7ae2b297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:09.450429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869b9d3-9f83-412b-be28-95cb7ae2b297 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:09.458389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 50/97] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:09 2022] GET /identity/v3/role_assignments?user.id=7db7540927364ffabd7480f25538841b&scope.project.id=e8ad380e71d24628bd90f6888c32b821&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:10.931933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27020312-f65c-4cb3-866d-ceb45c467fba None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:10.932442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27020312-f65c-4cb3-866d-ceb45c467fba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:10.932544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27020312-f65c-4cb3-866d-ceb45c467fba None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:10.934095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 48/98] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:10.939567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108bc67-1cb9-4746-afad-ab2a7f2a1874 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:10.940025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108bc67-1cb9-4746-afad-ab2a7f2a1874 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:10.940200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108bc67-1cb9-4746-afad-ab2a7f2a1874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:10.964935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f108bc67-1cb9-4746-afad-ab2a7f2a1874 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:10.966135 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f108bc67-1cb9-4746-afad-ab2a7f2a1874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:11.049635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 51/99] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:11.196402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc43ac-d901-4f9a-984e-572f7688285e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:11.196706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc43ac-d901-4f9a-984e-572f7688285e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:11.196883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc43ac-d901-4f9a-984e-572f7688285e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:11.198490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 49/100] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:11.203433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0035d-96b4-4ecf-8cb2-00a7c62fa895 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:11.204033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0035d-96b4-4ecf-8cb2-00a7c62fa895 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:11.204309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0035d-96b4-4ecf-8cb2-00a7c62fa895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:11.228758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6db0035d-96b4-4ecf-8cb2-00a7c62fa895 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:11.230001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6db0035d-96b4-4ecf-8cb2-00a7c62fa895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:11.248090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:11.314812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 52/101] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:11.504104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc43ac-d901-4f9a-984e-572f7688285e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:11.647527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc43ac-d901-4f9a-984e-572f7688285e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:11.653308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2c7ae-85bf-49bc-84d1-c5dd0d0b30ca None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:11.653884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2c7ae-85bf-49bc-84d1-c5dd0d0b30ca None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:11.654392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2c7ae-85bf-49bc-84d1-c5dd0d0b30ca None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:11.656029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 53/102] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:11.662005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba7cf7-387e-44e2-a711-7bddea2cc080 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:11.662565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba7cf7-387e-44e2-a711-7bddea2cc080 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:11.663045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba7cf7-387e-44e2-a711-7bddea2cc080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:11.681815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dbba7cf7-387e-44e2-a711-7bddea2cc080 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:11.683393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dbba7cf7-387e-44e2-a711-7bddea2cc080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:11.703436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:11.704445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:11.704933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:11.705408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:11.707458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:11.710162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:11.712828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 50/103] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:11.719885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:11.733838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:11.789543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 54/104] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:11.893367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba7cf7-387e-44e2-a711-7bddea2cc080 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:11.908294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87876f3-fe83-469c-bcc0-c10219ba0a5e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:11.982719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:11.983340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:11.983584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:11.983887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:11.984562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:11.986837 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:12.038382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:12.050238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 51/105] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:50:11 2022] POST /identity/v3/projects => generated 302 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:12.071787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba7cf7-387e-44e2-a711-7bddea2cc080 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:12.123946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:12.124989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.125477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.126017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.140872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:12.146429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:12.147364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:12.148267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:12.150752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 55/106] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:12.156812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:12.214056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:12.295483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:12.369717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab59ac-383f-49da-9af5-b16c72666a56 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:12.436666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3333e0-9a0b-4774-9382-cafebdb32754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:12.437732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3333e0-9a0b-4774-9382-cafebdb32754 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.439922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b245da-8a9f-42f9-8393-51f93cc2926b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:12.440578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3333e0-9a0b-4774-9382-cafebdb32754 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.441051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3333e0-9a0b-4774-9382-cafebdb32754 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.466830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3333e0-9a0b-4774-9382-cafebdb32754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-8d3333e0-9a0b-4774-9382-cafebdb32754 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:12.477164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:12.478054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:12.478807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:12.484879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 56/107] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:12.498321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166ad6-17bb-4a46-b5b2-3d9c561243ed None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.498683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166ad6-17bb-4a46-b5b2-3d9c561243ed None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.498963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166ad6-17bb-4a46-b5b2-3d9c561243ed None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.501050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 57/108] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:50:12.505553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166ad6-17bb-4a46-b5b2-3d9c561243ed None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:12.509223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:12.515997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca8585-88c1-4a2d-81a7-35acd98af6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:12.516952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8585-88c1-4a2d-81a7-35acd98af6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.516952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8585-88c1-4a2d-81a7-35acd98af6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.517310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8585-88c1-4a2d-81a7-35acd98af6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.518096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca8585-88c1-4a2d-81a7-35acd98af6f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:12.518489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca8585-88c1-4a2d-81a7-35acd98af6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:12.520709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca8585-88c1-4a2d-81a7-35acd98af6f2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:12.533211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 52/109] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:12.537321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfecbe0-2f64-4640-9c06-44a2976d0b3c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.540165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfecbe0-2f64-4640-9c06-44a2976d0b3c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.540665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfecbe0-2f64-4640-9c06-44a2976d0b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.566920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5bfecbe0-2f64-4640-9c06-44a2976d0b3c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:12.568629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5bfecbe0-2f64-4640-9c06-44a2976d0b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:12.636675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166ad6-17bb-4a46-b5b2-3d9c561243ed None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:12.654331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 53/110] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:12.655200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfecbe0-2f64-4640-9c06-44a2976d0b3c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:12.695292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9fd96-95e4-4a8a-884e-dcbfe5d87576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:12.696001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9fd96-95e4-4a8a-884e-dcbfe5d87576 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.696289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9fd96-95e4-4a8a-884e-dcbfe5d87576 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.696648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9fd96-95e4-4a8a-884e-dcbfe5d87576 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.697336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9fd96-95e4-4a8a-884e-dcbfe5d87576 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:12.697640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9fd96-95e4-4a8a-884e-dcbfe5d87576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:12.699145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9fd96-95e4-4a8a-884e-dcbfe5d87576 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:12.712267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:12.716796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 58/111] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:12.718238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c758b-38ae-498a-a9f9-10dfb1074251 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.718530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c758b-38ae-498a-a9f9-10dfb1074251 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.718820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c758b-38ae-498a-a9f9-10dfb1074251 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.729599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 59/112] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:12.730593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c758b-38ae-498a-a9f9-10dfb1074251 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:12.796480 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfecbe0-2f64-4640-9c06-44a2976d0b3c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:12.836912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2848a25-9e93-4ecd-8734-9dc3be22b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:12.837758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2848a25-9e93-4ecd-8734-9dc3be22b303 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.838052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2848a25-9e93-4ecd-8734-9dc3be22b303 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.838332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2848a25-9e93-4ecd-8734-9dc3be22b303 None admin] PATH_INFO: `/v3/users/b1f7f2689688441d94764a98d57a2913` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.856671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2848a25-9e93-4ecd-8734-9dc3be22b303 None admin] RBAC: Authorizing `identity:get_user(user_id=b1f7f2689688441d94764a98d57a2913)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:12.858155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2848a25-9e93-4ecd-8734-9dc3be22b303 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:12.860685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 54/113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:12 2022] GET /identity/v3/users/b1f7f2689688441d94764a98d57a2913 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:12.861864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286324e-a9a6-47fd-afae-161e8561b95c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.862154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286324e-a9a6-47fd-afae-161e8561b95c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.862434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286324e-a9a6-47fd-afae-161e8561b95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.868196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:12.883995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6286324e-a9a6-47fd-afae-161e8561b95c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:12.885754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6286324e-a9a6-47fd-afae-161e8561b95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:12.912772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c758b-38ae-498a-a9f9-10dfb1074251 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:12.950799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14653f-43f3-4052-8289-a53b32809b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:12.951588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14653f-43f3-4052-8289-a53b32809b7c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.952030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14653f-43f3-4052-8289-a53b32809b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.952411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14653f-43f3-4052-8289-a53b32809b7c None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.953644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14653f-43f3-4052-8289-a53b32809b7c None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:12.955231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14653f-43f3-4052-8289-a53b32809b7c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:12.957117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 60/114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:12 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:12.958443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3d994-189f-4ed5-a9f9-d3875eacdb29 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:12.958889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3d994-189f-4ed5-a9f9-d3875eacdb29 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:12.959237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3d994-189f-4ed5-a9f9-d3875eacdb29 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:12.960672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 61/115] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:12.961353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3d994-189f-4ed5-a9f9-d3875eacdb29 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:12.968353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 55/116] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:12.969055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286324e-a9a6-47fd-afae-161e8561b95c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.064256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:13.081070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3d994-189f-4ed5-a9f9-d3875eacdb29 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.114340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:13.117856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286324e-a9a6-47fd-afae-161e8561b95c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.121715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.122237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.122593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.122935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.161485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ecf26-4f62-4a72-b74f-268ac3ffa87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.163327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ecf26-4f62-4a72-b74f-268ac3ffa87b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.163787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ecf26-4f62-4a72-b74f-268ac3ffa87b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.164225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ecf26-4f62-4a72-b74f-268ac3ffa87b None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.165531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ecf26-4f62-4a72-b74f-268ac3ffa87b None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:13.167133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ecf26-4f62-4a72-b74f-268ac3ffa87b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:13.169656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 56/117] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:12 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:13.173705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193675-c539-437e-8215-958129beafa8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.174139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193675-c539-437e-8215-958129beafa8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.174241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193675-c539-437e-8215-958129beafa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.197389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2b193675-c539-437e-8215-958129beafa8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:13.198906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2b193675-c539-437e-8215-958129beafa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:13.202755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:13.204386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:13.278472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 57/118] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:13.279594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b193675-c539-437e-8215-958129beafa8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.322018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 62/119] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:13.328521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.328657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:13.369081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:13.426361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b193675-c539-437e-8215-958129beafa8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.465463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964804-dc43-4b46-974b-f5ff16deb724 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.467397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.468351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.468809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.469256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.481761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:13.488596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:13.489799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:13.490569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:13.493400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 58/120] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:13.494476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.510778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.511443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.511725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.512042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392/users/7db7540927364ffabd7480f25538841b/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.514931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f87007362fc43378dacfe7a12000392, user_id=7db7540927364ffabd7480f25538841b, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:13.516682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:13.540838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 63/121] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:13 2022] PUT /identity/v3/projects/7f87007362fc43378dacfe7a12000392/users/7db7540927364ffabd7480f25538841b/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:13.542352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.624360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:13.633666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29715e00-e5d5-4d8b-80b4-3c50884c6a6c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.673787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.674725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.675158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.675630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.687276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:13.690191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:13.695402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fc65-143d-48c5-8c68-7b8b626ee1e3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:13.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:13.697233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:13.698014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:13.701258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 59/122] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:13.702652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.745634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.746452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.746632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.747588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.748093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:13.749569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:13.751384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 64/123] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:13 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:13.752855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.830309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dec745-67a3-4df0-bb81-945f69e9087d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.868829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815330b8-159d-42ca-b414-26ef169e9c59 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:13.872572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.873451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.873939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.874412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.875281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:13.875789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:13.877456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:13.887337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 60/124] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:13.888516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.912155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:13.913113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:13.913814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:13.914215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:13.914500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:13.929483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:13.935788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:13.936684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:13.937499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 65/125] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:13.938144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:13.973204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:14.012921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daa931-9405-4c71-8f5b-826174eb35de None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.055874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.055874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.056128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.057329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.057329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:14.057329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.058904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.068575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 61/126] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.069430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.084342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf593aa1-2b87-4d75-b2ed-cf0f7560945b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.122974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.123627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.123933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.124210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.124983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] RBAC: Adding query filter params (role.id=6c47e35694694e9a9bbd645cae642089, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=b1f7f2689688441d94764a98d57a2913) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:14.125294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.126763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.132743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 66/127] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:13 2022] GET /identity/v3/role_assignments?user.id=b1f7f2689688441d94764a98d57a2913&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=6c47e35694694e9a9bbd645cae642089 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.133547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.220956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:14.244829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:14.261750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0d2e5-a5b3-478b-83e3-7d96fec1dfbb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.265489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c638306-3b65-477c-92e0-9cf0e81035ed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.334213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.335555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.336198 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.336740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.337046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.337316 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] PATH_INFO: `/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.337537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.337810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.338498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:14.339748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] RBAC: Authorizing `identity:get_user(user_id=dd261f057dda47fbbc28e2f5e9b2f3e7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.340277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.341338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.341731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.342989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 62/128] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:14 2022] GET /identity/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7 => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.343871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.359613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 67/129] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.360503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.486721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984e4b4-80c8-4456-86db-325e9148a7b9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.531935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.532890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.533799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.534140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.536068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.537922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.544019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 68/130] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:14 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.545815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.568239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:14.658917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a43a-b534-4152-9dfb-8de60f26a091 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.672231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbccfb3-260f-424f-9824-ec01add63989 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.713602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.714239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.714521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.714795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.715965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.716517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aff5c6-8c68-41df-af8b-637f1372d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.717154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aff5c6-8c68-41df-af8b-637f1372d557 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.717404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.717503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aff5c6-8c68-41df-af8b-637f1372d557 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.717704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aff5c6-8c68-41df-af8b-637f1372d557 None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.718837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aff5c6-8c68-41df-af8b-637f1372d557 None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.719009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 63/131] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:14 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.719713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.725496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aff5c6-8c68-41df-af8b-637f1372d557 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.727170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 69/132] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:14 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.728329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489279f-ee96-4187-a655-1d38b79ba8d5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.728603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489279f-ee96-4187-a655-1d38b79ba8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.728878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489279f-ee96-4187-a655-1d38b79ba8d5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.730076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 70/133] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:14.730773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489279f-ee96-4187-a655-1d38b79ba8d5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.751580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:14.848628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:14.848953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a463710-f41e-44eb-a513-3020d8aebb0c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.874255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489279f-ee96-4187-a655-1d38b79ba8d5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:14.889006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.889716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.890002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.890277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392/users/dd261f057dda47fbbc28e2f5e9b2f3e7/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.892227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f87007362fc43378dacfe7a12000392, user_id=dd261f057dda47fbbc28e2f5e9b2f3e7, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.895053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.912155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 64/134] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:14 2022] PUT /identity/v3/projects/7f87007362fc43378dacfe7a12000392/users/dd261f057dda47fbbc28e2f5e9b2f3e7/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:14.913038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:14.920157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1fa8-07c7-4aef-ab92-50d37bdf4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:14.921175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1fa8-07c7-4aef-ab92-50d37bdf4277 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.921664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1fa8-07c7-4aef-ab92-50d37bdf4277 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.922142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1fa8-07c7-4aef-ab92-50d37bdf4277 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.923864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb1fa8-07c7-4aef-ab92-50d37bdf4277 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:14.926419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb1fa8-07c7-4aef-ab92-50d37bdf4277 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:14.933803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 71/135] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:14 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:14.935473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec693d-9460-4605-83c9-aedcca3b91b0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:14.936130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec693d-9460-4605-83c9-aedcca3b91b0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:14.936656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec693d-9460-4605-83c9-aedcca3b91b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:14.958561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6aec693d-9460-4605-83c9-aedcca3b91b0 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:14.960237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6aec693d-9460-4605-83c9-aedcca3b91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:15.009931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:15.043112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407162-8230-4db2-b5e7-205cbfadad97 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:15.047511 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 72/136] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:15.052710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec693d-9460-4605-83c9-aedcca3b91b0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:15.084100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:15.085940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:15.086414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:15.087174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] PATH_INFO: `/v3/projects/e8ad380e71d24628bd90f6888c32b821` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:15.088697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ad380e71d24628bd90f6888c32b821)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:15.090536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:15.093526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 65/137] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:14 2022] GET /identity/v3/projects/e8ad380e71d24628bd90f6888c32b821 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:15.100905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:15.172045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:15.202525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec693d-9460-4605-83c9-aedcca3b91b0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:15.259023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:15.259929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:15.260376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:15.260705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392/users/7db7540927364ffabd7480f25538841b/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:15.265140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f87007362fc43378dacfe7a12000392, user_id=7db7540927364ffabd7480f25538841b, role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:15.266776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:15.295431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 73/138] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:15 2022] PUT /identity/v3/projects/7f87007362fc43378dacfe7a12000392/users/7db7540927364ffabd7480f25538841b/roles/2372f2ee163c430ca4b943c833677192 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:15.333695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2635d5-3855-47db-8126-2c12db134c15 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:15.375804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:15.376579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:15.376867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:15.377144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] PATH_INFO: `/v3/projects/e8ad380e71d24628bd90f6888c32b821/users/7db7540927364ffabd7480f25538841b/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:15.379689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ad380e71d24628bd90f6888c32b821, user_id=7db7540927364ffabd7480f25538841b, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:15.381373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:15.421630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 66/139] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:15 2022] PUT /identity/v3/projects/e8ad380e71d24628bd90f6888c32b821/users/7db7540927364ffabd7480f25538841b/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:15.491436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:15.641331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3b26-452c-4b6a-80cb-8dfbc23bc7a3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:15.683648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b8d17-c43b-4d82-8509-ec3097962c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:15.684646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8d17-c43b-4d82-8509-ec3097962c4c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:15.685120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8d17-c43b-4d82-8509-ec3097962c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:15.685583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8d17-c43b-4d82-8509-ec3097962c4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:15.687046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b8d17-c43b-4d82-8509-ec3097962c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:15.688723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b8d17-c43b-4d82-8509-ec3097962c4c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:15.690786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 74/140] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:15.696653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:15.704186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:15.819266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55beda-a01d-4a82-9200-95a1864f5b9c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:15.866176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bbba5-4337-48dd-bf84-44215c5c5e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:15.866809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bbba5-4337-48dd-bf84-44215c5c5e68 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:15.867148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bbba5-4337-48dd-bf84-44215c5c5e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:15.867442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bbba5-4337-48dd-bf84-44215c5c5e68 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:15.868189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bbba5-4337-48dd-bf84-44215c5c5e68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:15.869730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bbba5-4337-48dd-bf84-44215c5c5e68 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:15.904956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 67/141] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:50:15 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:16.648270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9c35c-3f88-495b-82a4-e0224f250af7 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:16.649303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9c35c-3f88-495b-82a4-e0224f250af7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:16.649779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9c35c-3f88-495b-82a4-e0224f250af7 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:16.651422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 75/142] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:16.657070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c19065-8941-4c05-9d3c-096f21bf17b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:16.658173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c19065-8941-4c05-9d3c-096f21bf17b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:16.658645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c19065-8941-4c05-9d3c-096f21bf17b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:16.684089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c9c19065-8941-4c05-9d3c-096f21bf17b5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:16.685624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c9c19065-8941-4c05-9d3c-096f21bf17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:16.762915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:16.782055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 68/143] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:17.185886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac36b8c-151b-4f96-95b4-0cc70dbf043a None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:17.186650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac36b8c-151b-4f96-95b4-0cc70dbf043a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:17.187145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac36b8c-151b-4f96-95b4-0cc70dbf043a None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:17.188708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 76/144] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:17.194890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106f4f-da8f-48f7-966a-1d4932a7d1a1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:17.195186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106f4f-da8f-48f7-966a-1d4932a7d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:17.195471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106f4f-da8f-48f7-966a-1d4932a7d1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:17.224880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-20106f4f-da8f-48f7-966a-1d4932a7d1a1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:17.226096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-20106f4f-da8f-48f7-966a-1d4932a7d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:17.309062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 69/145] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:17.346397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac36b8c-151b-4f96-95b4-0cc70dbf043a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:17.361955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:17.495183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac36b8c-151b-4f96-95b4-0cc70dbf043a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:17.539985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:17.540627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:17.540910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:17.541183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:17.555669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:17.560671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:17.561533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:17.562322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:17.564370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 77/146] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:17.570174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20106f4f-da8f-48f7-966a-1d4932a7d1a1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:17.707821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20106f4f-da8f-48f7-966a-1d4932a7d1a1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:17.749701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:17.755703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:17.756408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:17.756656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:17.756942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:17.757631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:17.757942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:17.759339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:17.772468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 70/147] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:17.780942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:17.892452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:17.942047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755395-f2bb-409a-88e1-f4a14d1c735a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:17.992305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:17.993014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:17.993298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:17.993577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:17.994841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:17.996433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:17.998216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 78/148] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:17 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:18.003873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:18.020284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:18.168214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686bad-e2b7-4ca6-87f2-7c24c9f51100 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:18.219903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576abec-7e6e-4bae-b17b-aa3ee7e8bbaa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:18.251271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005e4ff-d257-4186-9143-d1cedecffa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:18.252175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e4ff-d257-4186-9143-d1cedecffa7c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.252641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e4ff-d257-4186-9143-d1cedecffa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.253070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e4ff-d257-4186-9143-d1cedecffa7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.288663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f119ee-2d70-48b3-ae6e-a422414a9594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:18.289230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f119ee-2d70-48b3-ae6e-a422414a9594 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.289514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f119ee-2d70-48b3-ae6e-a422414a9594 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.289794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f119ee-2d70-48b3-ae6e-a422414a9594 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.293059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:18.332520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f119ee-2d70-48b3-ae6e-a422414a9594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-67f119ee-2d70-48b3-ae6e-a422414a9594 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:18.338169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:18.339030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:18.339807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 79/149] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:18.341055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d0e5-e2fb-4a34-8017-9eba2cb4bdfa None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.341055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d0e5-e2fb-4a34-8017-9eba2cb4bdfa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.341212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d0e5-e2fb-4a34-8017-9eba2cb4bdfa None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.342504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 80/150] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:18.343014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:18.348318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25d0e5-e2fb-4a34-8017-9eba2cb4bdfa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:18.368751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005e4ff-d257-4186-9143-d1cedecffa7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:18.370297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005e4ff-d257-4186-9143-d1cedecffa7c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:18.532835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 71/151] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:18.534160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ba67f-72be-4974-bd1c-92ab35129d25 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.534481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ba67f-72be-4974-bd1c-92ab35129d25 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.535253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ba67f-72be-4974-bd1c-92ab35129d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.568749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-056ba67f-72be-4974-bd1c-92ab35129d25 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:18.569881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-056ba67f-72be-4974-bd1c-92ab35129d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:18.572526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25d0e5-e2fb-4a34-8017-9eba2cb4bdfa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:18.648696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:18.649336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.649623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.649902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.650604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:18.650911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:18.652433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:18.679517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 81/152] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:18.680411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:18.689092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 72/153] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:18.696813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ba67f-72be-4974-bd1c-92ab35129d25 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:18.716228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:18.786835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:18.874157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946efe8-c04e-44af-a0c9-4d1c407df80f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:18.898671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ba67f-72be-4974-bd1c-92ab35129d25 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:18.952572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8d8d2-3d98-4705-b0a8-49c850c200b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:18.953231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8d8d2-3d98-4705-b0a8-49c850c200b6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.953521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8d8d2-3d98-4705-b0a8-49c850c200b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.953798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8d8d2-3d98-4705-b0a8-49c850c200b6 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.954957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8d8d2-3d98-4705-b0a8-49c850c200b6 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:18.956974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8d8d2-3d98-4705-b0a8-49c850c200b6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:18.959127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d205f-ce62-47af-adf9-43440329fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:18.959941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 82/154] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:18 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:18.960241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d205f-ce62-47af-adf9-43440329fa72 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.960719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d205f-ce62-47af-adf9-43440329fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.961122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffed39-fb75-4440-a2d0-b64feb86b4e3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.961386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d205f-ce62-47af-adf9-43440329fa72 None admin] PATH_INFO: `/v3/users/b1f7f2689688441d94764a98d57a2913` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.961482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffed39-fb75-4440-a2d0-b64feb86b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.961668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffed39-fb75-4440-a2d0-b64feb86b4e3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.966215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 83/155] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:18.967046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838811bd-1b2c-4afc-9893-52663a4de154 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.967429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d205f-ce62-47af-adf9-43440329fa72 None admin] RBAC: Authorizing `identity:get_user(user_id=b1f7f2689688441d94764a98d57a2913)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:18.967539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838811bd-1b2c-4afc-9893-52663a4de154 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.967723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838811bd-1b2c-4afc-9893-52663a4de154 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.969379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d205f-ce62-47af-adf9-43440329fa72 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:18.970040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 84/156] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:18.971333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa247a-b443-4029-ada5-265cedef69af None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.971618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa247a-b443-4029-ada5-265cedef69af None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.971989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa247a-b443-4029-ada5-265cedef69af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:18.973028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 73/157] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:18 2022] GET /identity/v3/users/b1f7f2689688441d94764a98d57a2913 => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:18.977220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c4d52-274f-44f2-a5bb-b8e739729bea None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:18.977770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c4d52-274f-44f2-a5bb-b8e739729bea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:18.978263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c4d52-274f-44f2-a5bb-b8e739729bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:19.006980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a63c4d52-274f-44f2-a5bb-b8e739729bea None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:19.008645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a63c4d52-274f-44f2-a5bb-b8e739729bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:19.016841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-aafa247a-b443-4029-ada5-265cedef69af None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:19.018026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aafa247a-b443-4029-ada5-265cedef69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:19.106658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 74/158] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:19.112811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c4d52-274f-44f2-a5bb-b8e739729bea None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:19.140302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:19.170143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 85/159] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:19.171105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa247a-b443-4029-ada5-265cedef69af None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:19.318558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c4d52-274f-44f2-a5bb-b8e739729bea None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:19.328835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:19.384550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:19.385190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:19.385456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:19.385734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:19.388532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=7db7540927364ffabd7480f25538841b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:19.388532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:19.388532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:19.395143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 75/160] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:19 2022] GET /identity/v3/role_assignments?user.id=7db7540927364ffabd7480f25538841b&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:19.400046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:19.450041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa247a-b443-4029-ada5-265cedef69af None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:19.504250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:19.546695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea2bf-9184-40a3-9172-d947be6c38b3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:19.558137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:19.558786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:19.559066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:19.559338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:19.568769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:19.570287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:19.572076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 86/161] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:19 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:19.580866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:19.604605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:19.605274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:19.605560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:19.605838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:19.621817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:19.627365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:19.628332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:19.629272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:19.632646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 76/162] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:19.638404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:19.804269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad376f16-6674-4e56-8520-1e3fb81bf8f7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:19.834995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2cbd7-64d1-442a-99f4-ff8602acd108 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:19.868205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:19.882058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:19.882735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:19.883017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:19.883727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:19.884657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:19.884983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:19.886389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:19.887928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:19.900302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 77/163] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:19.901211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:19.904623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:19.905310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:19.905588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:19.905860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392/users/b1f7f2689688441d94764a98d57a2913/roles/6c47e35694694e9a9bbd645cae642089` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:19.910442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f87007362fc43378dacfe7a12000392, user_id=b1f7f2689688441d94764a98d57a2913, role_id=6c47e35694694e9a9bbd645cae642089)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:19.911935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:19.953486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 87/164] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:19 2022] PUT /identity/v3/projects/7f87007362fc43378dacfe7a12000392/users/b1f7f2689688441d94764a98d57a2913/roles/6c47e35694694e9a9bbd645cae642089 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:19.954370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.021369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f331c6b-e594-4e59-b64d-262689005754 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.061007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.061575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.061909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.062188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.076403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:20.083374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:20.084340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:20.085115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 78/165] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:20.090669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.158310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:20.178964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f84a6-2e23-4310-91fb-020f3c32dcc7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.209596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32990c-0abb-409c-a2b1-2df5719fe2f1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.248498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.249155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.249439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.249717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] PATH_INFO: `/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.250886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] RBAC: Authorizing `identity:get_user(user_id=dd261f057dda47fbbc28e2f5e9b2f3e7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:20.252535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:20.255244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 79/166] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/users/dd261f057dda47fbbc28e2f5e9b2f3e7 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:20.255980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.257107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.257761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.258044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.258314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.289098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:20.298729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:20.299668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:20.300556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 88/167] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:20.301486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.376429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61548-8b58-4f49-a269-e3d28e07b2ad None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.414302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:20.421865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.422625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.422961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.424132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.424132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:20.424280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:20.427247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:20.429806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:20.438531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 80/168] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:20.445620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.532078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db64510-1bc3-4015-962b-d955dffb555b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.558425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903dc47-eaed-4e6e-bc30-2406f86f9cf4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.596077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.596524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ee8b-b1df-474b-9552-36716e63d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.596741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.597523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.597767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ee8b-b1df-474b-9552-36716e63d6db None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.597932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.598116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ee8b-b1df-474b-9552-36716e63d6db None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.598401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ee8b-b1df-474b-9552-36716e63d6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.598639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:20.598858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:20.600275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:20.609311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 81/169] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:20.610387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.666839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:20.707909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ee8b-b1df-474b-9552-36716e63d6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:20.709386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ee8b-b1df-474b-9552-36716e63d6db None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:20.724811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0001c88-20a6-4d96-9696-153b3117ff17 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.762278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.763039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.763422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.763780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.765049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:20.766591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:20.768555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 82/170] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:20.773887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.794471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:20.884972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22fa46-5865-4a3e-8ac7-d1b4b8878c2d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:20.886706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 89/171] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:20.896173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088d6c7-dc6e-4cdd-8958-8cae484f93ca None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.896503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088d6c7-dc6e-4cdd-8958-8cae484f93ca None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.896784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088d6c7-dc6e-4cdd-8958-8cae484f93ca None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.898190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 90/172] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:20.898987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088d6c7-dc6e-4cdd-8958-8cae484f93ca None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:20.908223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:20.946743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:20.947661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:20.949249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:20.949816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] PATH_INFO: `/v3/users/7db7540927364ffabd7480f25538841b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:20.951350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] RBAC: Authorizing `identity:get_user(user_id=7db7540927364ffabd7480f25538841b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:20.953856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:20.956496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 83/173] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/users/7db7540927364ffabd7480f25538841b => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:20.960287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:21.085745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08138e6c-6fa6-4584-b016-c859a997f16b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:21.102153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088d6c7-dc6e-4cdd-8958-8cae484f93ca None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:21.147723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cce78-5188-446e-87f7-03e5ac325695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:21.148385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cce78-5188-446e-87f7-03e5ac325695 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.148662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cce78-5188-446e-87f7-03e5ac325695 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.148935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cce78-5188-446e-87f7-03e5ac325695 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.151703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cce78-5188-446e-87f7-03e5ac325695 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:21.153173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cce78-5188-446e-87f7-03e5ac325695 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:21.154819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 84/174] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:21.160200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69906f-6f76-4f18-9cb9-9874ef175e0e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.160492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69906f-6f76-4f18-9cb9-9874ef175e0e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.160767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69906f-6f76-4f18-9cb9-9874ef175e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.167196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:21.167805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.168104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.168375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.199901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:21.207872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ad69906f-6f76-4f18-9cb9-9874ef175e0e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:21.209103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ad69906f-6f76-4f18-9cb9-9874ef175e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:21.288527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:21.308700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:21.310343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:21.320542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 85/175] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:21.321410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69906f-6f76-4f18-9cb9-9874ef175e0e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:21.453323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 91/176] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:21.454190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:21.480335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69906f-6f76-4f18-9cb9-9874ef175e0e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:21.502411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:21.524068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcfafd-7e6e-4819-8fa6-c28e8e228821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:21.524865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcfafd-7e6e-4819-8fa6-c28e8e228821 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.524999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcfafd-7e6e-4819-8fa6-c28e8e228821 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.525429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcfafd-7e6e-4819-8fa6-c28e8e228821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.562031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334955-d0f9-4942-a765-c1a0e0fdf13b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:21.586666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:21.601917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7c76-bc55-4db8-8eeb-4f7dc382fa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:21.602756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7c76-bc55-4db8-8eeb-4f7dc382fa06 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.603179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7c76-bc55-4db8-8eeb-4f7dc382fa06 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.603631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7c76-bc55-4db8-8eeb-4f7dc382fa06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.604634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7c76-bc55-4db8-8eeb-4f7dc382fa06 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=dd261f057dda47fbbc28e2f5e9b2f3e7) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:21.605128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7c76-bc55-4db8-8eeb-4f7dc382fa06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:21.606771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7c76-bc55-4db8-8eeb-4f7dc382fa06 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:21.607493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcfafd-7e6e-4819-8fa6-c28e8e228821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:21.609303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcfafd-7e6e-4819-8fa6-c28e8e228821 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:21.614006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 92/177] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:21 2022] GET /identity/v3/role_assignments?user.id=dd261f057dda47fbbc28e2f5e9b2f3e7&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:21.615359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0f6e-a7ca-4238-80a0-92f7602694d4 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.615787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0f6e-a7ca-4238-80a0-92f7602694d4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.616254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0f6e-a7ca-4238-80a0-92f7602694d4 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.617637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 93/178] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:21.623409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632f76b-26b5-408a-99d0-b3f001bda52f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.624012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632f76b-26b5-408a-99d0-b3f001bda52f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.624452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632f76b-26b5-408a-99d0-b3f001bda52f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.626256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 94/179] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:21.627210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632f76b-26b5-408a-99d0-b3f001bda52f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:21.753523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 86/180] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:21.754374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632f76b-26b5-408a-99d0-b3f001bda52f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:21.760667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333a93d-dde5-4fb8-96c0-b5796e622cea None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.764325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333a93d-dde5-4fb8-96c0-b5796e622cea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.764722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333a93d-dde5-4fb8-96c0-b5796e622cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.770257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:21.793663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6333a93d-dde5-4fb8-96c0-b5796e622cea None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:21.795144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6333a93d-dde5-4fb8-96c0-b5796e622cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:21.804548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:21.805244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.805528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.805801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] PATH_INFO: `/v3/projects/e8ad380e71d24628bd90f6888c32b821` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.807453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ad380e71d24628bd90f6888c32b821)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:21.809016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:21.812705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 95/181] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:21 2022] GET /identity/v3/projects/e8ad380e71d24628bd90f6888c32b821 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:21.812705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:21.878578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 87/182] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:21.880804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b334e-785f-459a-a958-96cbf67da27a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.881299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b334e-785f-459a-a958-96cbf67da27a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.881789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b334e-785f-459a-a958-96cbf67da27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.901299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e73b334e-785f-459a-a958-96cbf67da27a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:21.902605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e73b334e-785f-459a-a958-96cbf67da27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:21.911540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:21.928885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564df488-2331-46c0-b672-f2d3091b3ec4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:21.967368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:21.968300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:21.968676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:21.969120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:21.981040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 88/183] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:21.981322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:21.982258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b334e-785f-459a-a958-96cbf67da27a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:21.998106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:21.999038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 96/184] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:22.000826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.093634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:22.128874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b334e-785f-459a-a958-96cbf67da27a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.180542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.181574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.182095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.182581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.184026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.185562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.187300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 89/185] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:21 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:22.192033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.208781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:22.221206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dabea1-2dbd-44e3-9f90-adb574858f66 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.256901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.257509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.257789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.258060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.258785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=e8ad380e71d24628bd90f6888c32b821, user.id=7db7540927364ffabd7480f25538841b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:22.259090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.260584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.267164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 97/186] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:21 2022] GET /identity/v3/role_assignments?user.id=7db7540927364ffabd7480f25538841b&scope.project.id=e8ad380e71d24628bd90f6888c32b821&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:22.268040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.323002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6658933-d83d-4ca4-a499-a13e8b4c09ef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.367503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.368601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.369081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.369593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.370465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:22.370953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.372574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.383801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 90/187] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:22.385744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:22.386849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.391212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48032a50-dfde-4ba1-884d-80944e9f7855 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.436742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.436974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.437261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.437541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.438283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] RBAC: Adding query filter params (role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=7db7540927364ffabd7480f25538841b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:22.438592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.440095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.447932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 98/188] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/role_assignments?user.id=7db7540927364ffabd7480f25538841b&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=2372f2ee163c430ca4b943c833677192 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:22.456795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.484976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:22.526872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9bbd8-32ea-4cf7-b722-0c520aa05924 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.572340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.573493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.573869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.574239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.590211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:22.595996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:22.597701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:22.598725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:22.601642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 91/189] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:22.607701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.618474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f885bd4-9b51-4920-833d-9203c9e1cf64 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.660992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.661414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.661801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.662169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.666007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.667702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.669609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 99/190] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:22.670480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.714080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:22.745242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48718e4c-e801-4a83-946a-0d18c481b310 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.784196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:22.796726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.797515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.797897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.800508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] PATH_INFO: `/v3/users/1f1b7263ee6e467b942bcf534c7f8f99` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.812164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b72f0-a4e8-4c6a-b82e-2c6b50c71e42 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.822630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1b7263ee6e467b942bcf534c7f8f99)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.822630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.823977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 92/191] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/users/1f1b7263ee6e467b942bcf534c7f8f99 => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:22.824819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.849908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.850513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.850792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.851063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.851729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:22.852064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.853439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.861415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 100/192] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:22.862192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:22.928943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab14038-01da-4634-bf46-e5f4b6fcc782 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.956857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:22.962334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06dce1-d109-4e30-80ac-524e195292ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:22.962938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06dce1-d109-4e30-80ac-524e195292ef None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.963215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06dce1-d109-4e30-80ac-524e195292ef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.963485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06dce1-d109-4e30-80ac-524e195292ef None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.963750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d259be-8d92-4690-8f19-e3137d930fb4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:22.964158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06dce1-d109-4e30-80ac-524e195292ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:22.965616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06dce1-d109-4e30-80ac-524e195292ef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:22.986879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 93/193] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:50:22 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:22.989771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056b58-78e8-4795-a790-51f59feb5a12 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:22.990084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056b58-78e8-4795-a790-51f59feb5a12 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:22.990510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056b58-78e8-4795-a790-51f59feb5a12 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:22.995944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 94/194] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:22.995944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90056b58-78e8-4795-a790-51f59feb5a12 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:23.000603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:23.001230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:23.001510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:23.001778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:23.059431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:23.084138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:23.085661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:23.123127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90056b58-78e8-4795-a790-51f59feb5a12 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:23.165270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71682b7f-10af-42cb-acb9-34e8edb2a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:23.165917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71682b7f-10af-42cb-acb9-34e8edb2a136 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:23.166613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71682b7f-10af-42cb-acb9-34e8edb2a136 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:23.166895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71682b7f-10af-42cb-acb9-34e8edb2a136 None admin] PATH_INFO: `/v3/users/1f1b7263ee6e467b942bcf534c7f8f99` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:23.170216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71682b7f-10af-42cb-acb9-34e8edb2a136 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1b7263ee6e467b942bcf534c7f8f99)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:23.170216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71682b7f-10af-42cb-acb9-34e8edb2a136 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:23.171864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 95/195] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/users/1f1b7263ee6e467b942bcf534c7f8f99 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:23.172888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304cc0-0aa7-4e92-ab9b-2033bfc3a457 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:23.173167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304cc0-0aa7-4e92-ab9b-2033bfc3a457 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:23.173437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304cc0-0aa7-4e92-ab9b-2033bfc3a457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:23.192673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:23.193565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-af304cc0-0aa7-4e92-ab9b-2033bfc3a457 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:23.194781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af304cc0-0aa7-4e92-ab9b-2033bfc3a457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:23.200622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 101/196] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:23.201459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:23.273622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 96/197] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:23.274535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af304cc0-0aa7-4e92-ab9b-2033bfc3a457 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:23.339032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a621d-9fa0-4a00-9de2-3be4292418f6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:23.349888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:23.396050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:23.399113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:23.399113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:23.399113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:23.461009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af304cc0-0aa7-4e92-ab9b-2033bfc3a457 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:23.525313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:23.529324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:23.529740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:23.530041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:23.530319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:23.531524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:23.533266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:23.540022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 97/198] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:23 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:23.545179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:23.557978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:23.558672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:23.766074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 102/199] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:23.768240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:23.792816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3b5b7-12ca-4d43-9f67-ca89e7de5082 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:23.821998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:23.852495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:23.853570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:23.854439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:23.854908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:23.856147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=1f1b7263ee6e467b942bcf534c7f8f99) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:23.856653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:23.858599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:23.868210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 98/200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:23 2022] GET /identity/v3/role_assignments?user.id=1f1b7263ee6e467b942bcf534c7f8f99&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:23.869450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:23.991054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:23.994500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4903-6e2d-456a-8e6d-3b26fd6e876a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:24.014818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf7b75-6136-4f10-9343-7d2a4c7b8869 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:24.041434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:24.042002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:24.042279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:24.042550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:24.044491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:24.045951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:24.047966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 99/201] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:23 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:24.054264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:24.107199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:24.125099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:24.125750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:24.126025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:24.126291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:24.160757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:24.165661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:24.166627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:24.167909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:24.168541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c9ab5-5ea4-4c8f-87db-32fb5ae95223 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:24.168613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 103/202] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:24.173604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:24.220960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:24.221616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:24.221952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:24.222242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:24.223084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] RBAC: Adding query filter params (role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=1f1b7263ee6e467b942bcf534c7f8f99) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:24.223393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:24.224916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:24.232362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 100/203] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:24 2022] GET /identity/v3/role_assignments?user.id=1f1b7263ee6e467b942bcf534c7f8f99&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=2372f2ee163c430ca4b943c833677192 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:24.346340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c96591-86da-4f45-b091-fc5fdec600a8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:24.413365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:24.414444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:24.414945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:24.415452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:24.416415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:24.416993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:24.418707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:24.433080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 104/204] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:24.439086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:24.603305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff6f6-b409-4261-9dd3-574effb304f5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:24.612280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:24.639127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:24.639769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:24.640074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:24.640346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] PATH_INFO: `/v3/users/b1f7f2689688441d94764a98d57a2913` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:24.641479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] RBAC: Authorizing `identity:get_user(user_id=b1f7f2689688441d94764a98d57a2913)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:24.642916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:24.645629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 101/205] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:24 2022] GET /identity/v3/users/b1f7f2689688441d94764a98d57a2913 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:24.651102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:24.670196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:24.779662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751c56e-efe3-4ebd-a4f7-6f77d70b4b11 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:24.826232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:24.826842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:24.827119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:24.827392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:24.902222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:24.903637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:24.922115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:25.039470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 105/206] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:25.044721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:25.215948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079f373-7403-4cf0-908f-6b3aa95ceb96 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:25.267227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93146652-279b-4fe3-8340-29ffb70af784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:25.268017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93146652-279b-4fe3-8340-29ffb70af784 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:25.268302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93146652-279b-4fe3-8340-29ffb70af784 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:25.269135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93146652-279b-4fe3-8340-29ffb70af784 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:25.276911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93146652-279b-4fe3-8340-29ffb70af784 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:25.278440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93146652-279b-4fe3-8340-29ffb70af784 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:25.280676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 102/207] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:25 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:25.289029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:25.470689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d07737-ce08-4e06-9a84-d6df4dbb4196 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:25.492219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:25.522440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57817eed-5529-4d6a-97b7-be06caf760a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:25.523056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57817eed-5529-4d6a-97b7-be06caf760a4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:25.523340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57817eed-5529-4d6a-97b7-be06caf760a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:25.523615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57817eed-5529-4d6a-97b7-be06caf760a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:25.524376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57817eed-5529-4d6a-97b7-be06caf760a4 None admin] RBAC: Adding query filter params (role.id=6c47e35694694e9a9bbd645cae642089, scope.project.id=7f87007362fc43378dacfe7a12000392, user.id=b1f7f2689688441d94764a98d57a2913) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:25.524684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57817eed-5529-4d6a-97b7-be06caf760a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:25.526131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57817eed-5529-4d6a-97b7-be06caf760a4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:25.536964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 106/208] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:25 2022] GET /identity/v3/role_assignments?user.id=b1f7f2689688441d94764a98d57a2913&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=6c47e35694694e9a9bbd645cae642089 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:25.838393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375da8f2-fa97-4635-b0d2-c252c326a8e5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:25.838760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375da8f2-fa97-4635-b0d2-c252c326a8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:25.838955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375da8f2-fa97-4635-b0d2-c252c326a8e5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:25.840445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 103/209] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:25.846982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13910b2c-991d-4b32-9327-503acf732934 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:25.847276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13910b2c-991d-4b32-9327-503acf732934 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:25.847550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13910b2c-991d-4b32-9327-503acf732934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:25.868942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-13910b2c-991d-4b32-9327-503acf732934 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:25.870515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-13910b2c-991d-4b32-9327-503acf732934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:25.945193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 107/210] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:26.212909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246671-237e-4d4e-86aa-dc727209bf2b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.213393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246671-237e-4d4e-86aa-dc727209bf2b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.213393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246671-237e-4d4e-86aa-dc727209bf2b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.214985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 104/211] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:26.220509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d840c-4b70-40b3-81fe-e966e0c2835c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.220926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d840c-4b70-40b3-81fe-e966e0c2835c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.221115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d840c-4b70-40b3-81fe-e966e0c2835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.240748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95246671-237e-4d4e-86aa-dc727209bf2b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:26.243337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d67d840c-4b70-40b3-81fe-e966e0c2835c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:26.244923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d67d840c-4b70-40b3-81fe-e966e0c2835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:26.313211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:26.334875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 108/212] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:26.366490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9024fa-aee1-4e18-ad90-ce48491739ce None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.366825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9024fa-aee1-4e18-ad90-ce48491739ce None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.367039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9024fa-aee1-4e18-ad90-ce48491739ce None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.368571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 109/213] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:50:26.373339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192674a9-e4a1-4e49-a500-881b442313b5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.373617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192674a9-e4a1-4e49-a500-881b442313b5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.373891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192674a9-e4a1-4e49-a500-881b442313b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.377050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95246671-237e-4d4e-86aa-dc727209bf2b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:26.395605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-192674a9-e4a1-4e49-a500-881b442313b5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:26.396811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-192674a9-e4a1-4e49-a500-881b442313b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:26.419031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:26.419813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.420167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.420445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.433147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:26.439763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:26.441506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:26.442295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 105/214] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:26.448734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:26.451400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:26.479522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 110/215] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:26.558960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6c76c-1a34-458d-af35-b27cd6c37371 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:26.561529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:26.595089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:26.595702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.596014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.596288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.597569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:26.597872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:26.599257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:26.607791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 106/216] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:26.613579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192674a9-e4a1-4e49-a500-881b442313b5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:26.627616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:26.749645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97122ec-e157-4236-ad0e-925e406c2dfc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:26.753998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192674a9-e4a1-4e49-a500-881b442313b5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:26.786248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7f203-399b-482a-969d-cf5d67eb99b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:26.786882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f203-399b-482a-969d-cf5d67eb99b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.787165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f203-399b-482a-969d-cf5d67eb99b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.787449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f203-399b-482a-969d-cf5d67eb99b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.788574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7f203-399b-482a-969d-cf5d67eb99b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:26.790030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7f203-399b-482a-969d-cf5d67eb99b2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:26.790940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:26.791575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.791668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 107/217] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:26.792897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d25b4-4cd2-4c58-80b0-c33b36213c58 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.793187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d25b4-4cd2-4c58-80b0-c33b36213c58 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.793465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d25b4-4cd2-4c58-80b0-c33b36213c58 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.794661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 108/218] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:26.795362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d25b4-4cd2-4c58-80b0-c33b36213c58 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:26.796345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.796647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] PATH_INFO: `/v3/users/ae5aad576afe4003b431336da0242306` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.811639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] RBAC: Authorizing `identity:get_user(user_id=ae5aad576afe4003b431336da0242306)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:26.813151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:26.818152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 111/219] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:26 2022] GET /identity/v3/users/ae5aad576afe4003b431336da0242306 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:26.818152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:26.831434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:26.843146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:26.897943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d25b4-4cd2-4c58-80b0-c33b36213c58 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:26.916182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7453-c480-4bd2-84f7-5ca9f7186360 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:26.931790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17ca2c-b950-4fc8-a954-65e1e5f2c466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:26.932449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ca2c-b950-4fc8-a954-65e1e5f2c466 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.932732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ca2c-b950-4fc8-a954-65e1e5f2c466 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.933011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ca2c-b950-4fc8-a954-65e1e5f2c466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.943383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17ca2c-b950-4fc8-a954-65e1e5f2c466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fb17ca2c-b950-4fc8-a954-65e1e5f2c466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:26.948646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:26.949597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:26.950421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:26.951027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:26.951121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 109/220] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:26.951190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.951190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.951190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.951354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee68d10-5ac4-4761-809e-1a4958259f22 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:26.951428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee68d10-5ac4-4761-809e-1a4958259f22 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:26.951830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:26.951944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee68d10-5ac4-4761-809e-1a4958259f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:26.953053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:26.970095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4ee68d10-5ac4-4761-809e-1a4958259f22 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:26.972267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4ee68d10-5ac4-4761-809e-1a4958259f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:26.975895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 112/221] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:50:26 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:26.976774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.055939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 110/222] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:27.064933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee68d10-5ac4-4761-809e-1a4958259f22 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.077639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.096542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f06c4-26fd-41bf-89b1-28c77666485a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.104862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.138797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.139534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.139823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.140145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.205072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee68d10-5ac4-4761-809e-1a4958259f22 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.227809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:27.232104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:27.256037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.257118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.257407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.257682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.258356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:27.258770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:27.260075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:27.272994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 111/223] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:27.278428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.338478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.362891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 113/224] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:27.363659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.391775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.410249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292f454-b01a-4584-bcef-49e5906249ae None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.456967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.457789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.458135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.458479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] PATH_INFO: `/v3/users/1f1b7263ee6e467b942bcf534c7f8f99` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.460395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1b7263ee6e467b942bcf534c7f8f99)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:27.461942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:27.467958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 112/225] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/users/1f1b7263ee6e467b942bcf534c7f8f99 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:27.469003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.496941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61f827-90ab-4ffe-b74a-88336fdda1de None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.545514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.546657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.547452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.548231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.562515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:27.569605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:27.570541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:27.571334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:27.574035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 114/226] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:27.575374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.592873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4af4bb-ecee-4bc6-9788-960cc43cbcd3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.636274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.636898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.637173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.637445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.638575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:27.640355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.640540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:27.644255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 113/227] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:27.645051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.676555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.719058 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06c5a3-67da-489e-a71e-59af2a5fe845 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.762691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.763372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.763654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.763952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.765181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:27.766744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:27.768568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 115/228] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:27.769882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b84cc7-6abf-463d-8b34-e2542b19c730 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.774139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.813760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.814960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.816613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.817248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.818214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:27.818717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:27.820343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:27.829630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 114/229] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:27.831969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.889571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e918c5-1875-4550-9f0a-9a4e2bce11ac None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:27.903594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.933621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:27.934624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:27.935428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:27.935937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:27.937168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=ae5aad576afe4003b431336da0242306) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:27.937913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:27.939725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:27.950770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:27.952746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 116/230] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/role_assignments?user.id=ae5aad576afe4003b431336da0242306&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:27.953689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:27.976962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca329cd3-f626-417f-a6a3-3ba05c0edaae None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:28.026587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:28.027499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:28.027973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:28.028388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25/users/1f1b7263ee6e467b942bcf534c7f8f99/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:28.030479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] RBAC: Authorizing `identity:create_grant(project_id=994f81afb7594f73a78a9d847a544f25, user_id=1f1b7263ee6e467b942bcf534c7f8f99, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:28.032382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:28.075069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 115/231] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:27 2022] PUT /identity/v3/projects/994f81afb7594f73a78a9d847a544f25/users/1f1b7263ee6e467b942bcf534c7f8f99/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:28.109158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc6cc-d44b-433a-8eeb-7a6f5fe0b806 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:28.166760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:28.167339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:28.167623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:28.167925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] PATH_INFO: `/v3/users/1f1b7263ee6e467b942bcf534c7f8f99` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:28.169194 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1b7263ee6e467b942bcf534c7f8f99)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:28.171038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:28.172866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 117/232] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:27 2022] GET /identity/v3/users/1f1b7263ee6e467b942bcf534c7f8f99 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:28.179653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:28.337863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04639b-a9d1-4d3d-9254-f40b31c70f11 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:28.374484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53663a8-ef33-4d32-b370-73490eea201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:28.375552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53663a8-ef33-4d32-b370-73490eea201e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:28.376050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53663a8-ef33-4d32-b370-73490eea201e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:28.376517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53663a8-ef33-4d32-b370-73490eea201e None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:28.378585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53663a8-ef33-4d32-b370-73490eea201e None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:28.380311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53663a8-ef33-4d32-b370-73490eea201e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:28.382562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 116/233] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:28 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:28.390215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:28.466637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:28.536451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e0fc4-b77a-453e-b932-f7818a49e5d5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:28.591056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fd779-de09-4d45-9e27-afdd3a4be3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:28.591734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fd779-de09-4d45-9e27-afdd3a4be3a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:28.592041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fd779-de09-4d45-9e27-afdd3a4be3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:28.592314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fd779-de09-4d45-9e27-afdd3a4be3a3 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25/users/1f1b7263ee6e467b942bcf534c7f8f99/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:28.594567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fd779-de09-4d45-9e27-afdd3a4be3a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=994f81afb7594f73a78a9d847a544f25, user_id=1f1b7263ee6e467b942bcf534c7f8f99, role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:28.596067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fd779-de09-4d45-9e27-afdd3a4be3a3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:28.614101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 118/234] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:28 2022] PUT /identity/v3/projects/994f81afb7594f73a78a9d847a544f25/users/1f1b7263ee6e467b942bcf534c7f8f99/roles/2372f2ee163c430ca4b943c833677192 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:29.552085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb9ed7-94fe-4dd6-8169-5670982480eb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:29.553244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb9ed7-94fe-4dd6-8169-5670982480eb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:29.553732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb9ed7-94fe-4dd6-8169-5670982480eb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:29.555461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 117/235] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:29.561619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2fbe7-dab1-47f6-8bc3-be73e0c6d5bd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:29.562152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2fbe7-dab1-47f6-8bc3-be73e0c6d5bd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:29.562152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2fbe7-dab1-47f6-8bc3-be73e0c6d5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:29.597387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-81b2fbe7-dab1-47f6-8bc3-be73e0c6d5bd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:29.598672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-81b2fbe7-dab1-47f6-8bc3-be73e0c6d5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:29.665575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851d8b1-cae4-4555-bc5f-6e6528fb0467 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:29.665776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851d8b1-cae4-4555-bc5f-6e6528fb0467 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:29.666051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851d8b1-cae4-4555-bc5f-6e6528fb0467 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:29.667658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 118/236] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:29.672793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26773d3-1067-4835-bdcc-1f9a20d5de57 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:29.673078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26773d3-1067-4835-bdcc-1f9a20d5de57 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:29.673355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26773d3-1067-4835-bdcc-1f9a20d5de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:29.697397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 119/237] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:29.708913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b26773d3-1067-4835-bdcc-1f9a20d5de57 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:29.710140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b26773d3-1067-4835-bdcc-1f9a20d5de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:29.718242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:29.810083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 119/238] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:29.960991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2fbe7-dab1-47f6-8bc3-be73e0c6d5bd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:30.005710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:30.091094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2fbe7-dab1-47f6-8bc3-be73e0c6d5bd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:30.144628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:30.145277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.145557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.145828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.156909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:30.163063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:30.164009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:30.164812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 120/239] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:30.171376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26773d3-1067-4835-bdcc-1f9a20d5de57 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:30.282416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26773d3-1067-4835-bdcc-1f9a20d5de57 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:30.323275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:30.323911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.324196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.324470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.325145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:30.325448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:30.326835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:30.336013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 120/240] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:30.344907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:30.352990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:30.356725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:30.504527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70729ff3-88b2-400a-9baa-83cf8993f5f1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:30.541625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:30.542656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.543133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.543595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.545230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:30.546964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:30.548902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 121/241] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:30.554263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:30.556363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:30.576244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54ba0c-06f9-4318-9bbe-f954b3ec5906 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:30.632553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:30.633197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.633478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.633753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] PATH_INFO: `/v3/users/90fcb15b0a7145a1a4216a0ff545d0d1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.649826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:30.671027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] RBAC: Authorizing `identity:get_user(user_id=90fcb15b0a7145a1a4216a0ff545d0d1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:30.672876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:30.674728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 121/242] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:30 2022] GET /identity/v3/users/90fcb15b0a7145a1a4216a0ff545d0d1 => generated 297 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:30.680238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:30.686503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0350-2fb2-4c3c-9b34-1df0a31cce6b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:30.724595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223b2dd-3f09-455c-8b56-e6af59c371c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:30.725207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223b2dd-3f09-455c-8b56-e6af59c371c5 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.725485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223b2dd-3f09-455c-8b56-e6af59c371c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.725757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223b2dd-3f09-455c-8b56-e6af59c371c5 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.726407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223b2dd-3f09-455c-8b56-e6af59c371c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:30.727850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223b2dd-3f09-455c-8b56-e6af59c371c5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:30.754995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 122/243] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:50:30 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:30.757140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5404b-b82e-4ea2-9e95-128943623bf7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.757503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5404b-b82e-4ea2-9e95-128943623bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.757780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5404b-b82e-4ea2-9e95-128943623bf7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.759293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 123/244] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:30.765584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3fdfe-a95f-4025-8021-2e4c53d7b769 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.765909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3fdfe-a95f-4025-8021-2e4c53d7b769 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.766134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3fdfe-a95f-4025-8021-2e4c53d7b769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.815606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-78a3fdfe-a95f-4025-8021-2e4c53d7b769 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:30.817566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78a3fdfe-a95f-4025-8021-2e4c53d7b769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:30.825870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5aab-9cb5-46e3-94fc-4b6857130b22 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:30.898077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106bca1-01ee-4c7f-9bad-de947937b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:30.898702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106bca1-01ee-4c7f-9bad-de947937b8dd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:30.898979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106bca1-01ee-4c7f-9bad-de947937b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:30.899246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106bca1-01ee-4c7f-9bad-de947937b8dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:30.940316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:30.964800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 124/245] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:31.001419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106bca1-01ee-4c7f-9bad-de947937b8dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:31.003229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106bca1-01ee-4c7f-9bad-de947937b8dd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:31.122656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 122/246] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:31.129106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3fdfe-a95f-4025-8021-2e4c53d7b769 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:31.169663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044a19-07a3-4d93-9aae-36ba0fd19dca None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.170002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044a19-07a3-4d93-9aae-36ba0fd19dca None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.170209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044a19-07a3-4d93-9aae-36ba0fd19dca None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.171902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 123/247] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:50:31.179393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ef6e2-945e-4edb-abc7-5d82dab978c2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.179973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ef6e2-945e-4edb-abc7-5d82dab978c2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.180471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ef6e2-945e-4edb-abc7-5d82dab978c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.197688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:31.201906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-065ef6e2-945e-4edb-abc7-5d82dab978c2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:31.203304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-065ef6e2-945e-4edb-abc7-5d82dab978c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:31.267593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3fdfe-a95f-4025-8021-2e4c53d7b769 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:31.278348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:31.312290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 124/248] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:31.317656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:31.318364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.318644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.318918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.320319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:31.321932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:31.323993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 125/249] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:31 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:31.329075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ef6e2-945e-4edb-abc7-5d82dab978c2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:31.329384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:31.469491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca9ee0-6cf3-4884-9174-0b8c271bc1e0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:31.497420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:31.498808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ef6e2-945e-4edb-abc7-5d82dab978c2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:31.508103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330097-2833-44ff-9836-4c48948454b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:31.508722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330097-2833-44ff-9836-4c48948454b8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.509003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330097-2833-44ff-9836-4c48948454b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.509329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330097-2833-44ff-9836-4c48948454b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.510122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15330097-2833-44ff-9836-4c48948454b8 None admin] RBAC: Adding query filter params (role.id=6c47e35694694e9a9bbd645cae642089, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=90fcb15b0a7145a1a4216a0ff545d0d1) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:31.510441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15330097-2833-44ff-9836-4c48948454b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:31.511920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15330097-2833-44ff-9836-4c48948454b8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:31.518215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 126/250] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:31 2022] GET /identity/v3/role_assignments?user.id=90fcb15b0a7145a1a4216a0ff545d0d1&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=6c47e35694694e9a9bbd645cae642089 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:31.528057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6b61a-a7ee-49d3-986d-d558735e1efb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.528485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6b61a-a7ee-49d3-986d-d558735e1efb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.528775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6b61a-a7ee-49d3-986d-d558735e1efb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.530329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 127/251] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:31 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:31.536646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989796d-ce74-422f-a68a-3530f064df6f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.536936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989796d-ce74-422f-a68a-3530f064df6f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.537205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989796d-ce74-422f-a68a-3530f064df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.543154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:31.543816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.544132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.544515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.557506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:31.563926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:31.565671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:31.566605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 125/252] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:31.571865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:31.581614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0989796d-ce74-422f-a68a-3530f064df6f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:31.583552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0989796d-ce74-422f-a68a-3530f064df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:31.632394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:31.709703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4b26-3195-4c66-adce-14f038d89a8a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:31.712833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 128/253] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:31.730364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989796d-ce74-422f-a68a-3530f064df6f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:31.752491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:31.753577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.753857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.754126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.754789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:31.755088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:31.756575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:31.767695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 126/254] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:31.773872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:31.863584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989796d-ce74-422f-a68a-3530f064df6f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:31.909518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:31.943347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:31.944019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:31.944296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:31.944565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:31.948963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0369e-08c7-4783-ad4f-7bada1787805 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:31.955163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:31.974352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:31.980661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:31.981547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:31.982478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:31.986007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 129/255] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:31.992805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.028118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.029130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.029611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.030078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] PATH_INFO: `/v3/users/ae5aad576afe4003b431336da0242306` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:32.031499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] RBAC: Authorizing `identity:get_user(user_id=ae5aad576afe4003b431336da0242306)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:32.035255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:32.037106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 127/256] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:31 2022] GET /identity/v3/users/ae5aad576afe4003b431336da0242306 => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:32.042848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.166012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c74118-61e6-4fd7-8367-5d6f48d2c4c7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:32.198698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7291028-c095-4e36-8946-53c66cb5594b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:32.235690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.237165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.237624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.238043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:32.239375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:32.241035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:32.243152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 128/257] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:32 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:32.248441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.260570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.261376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.261518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.261744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:32.262427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:32.262734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:32.264149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:32.279094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 130/258] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:32.298758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.305056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:32.370993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c71b9-8837-4d8b-9edd-7e79fbb392f2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:32.410252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.411247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.411671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.412112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25/users/ae5aad576afe4003b431336da0242306/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:32.414243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] RBAC: Authorizing `identity:create_grant(project_id=994f81afb7594f73a78a9d847a544f25, user_id=ae5aad576afe4003b431336da0242306, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:32.416032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:32.426983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:32.436768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 129/259] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:32 2022] PUT /identity/v3/projects/994f81afb7594f73a78a9d847a544f25/users/ae5aad576afe4003b431336da0242306/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 10:50:32.439547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.559255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb0b04-252c-41a0-922d-6e5c1817dd40 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:32.640285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.640967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.641217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.641922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:32.655218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:32.661793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:32.662775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:32.663679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 131/260] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:32.670707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.675258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d0736-bf7f-4ec3-be5b-c5178ae59619 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:32.688788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:32.716596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.717287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.717576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.717863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] PATH_INFO: `/v3/users/1f1b7263ee6e467b942bcf534c7f8f99` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:32.719532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1b7263ee6e467b942bcf534c7f8f99)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:32.722210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:32.724929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 130/261] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:32 2022] GET /identity/v3/users/1f1b7263ee6e467b942bcf534c7f8f99 => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:32.731156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.875694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c541-3375-4ceb-935e-8cdfb146d44a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:32.878114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:32.911557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74e0db-16c0-4356-9bab-7330e2d62069 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:32.949941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.950560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.950847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.951124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:32.951807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:32.952133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:32.953511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:32.970323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 132/262] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:32.975604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:32.979118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:32.979734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:32.980037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:32.980310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:33.024233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:33.068699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:33.070174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:33.249065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 131/263] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:33.251985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0dcee-f6f0-4fe6-a351-a4146fda7a0f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:33.254995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:33.321532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:33.322571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:33.323057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:33.323532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] PATH_INFO: `/v3/users/1f1b7263ee6e467b942bcf534c7f8f99` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:33.328660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1b7263ee6e467b942bcf534c7f8f99)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:33.331044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:33.333090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 133/264] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:32 2022] GET /identity/v3/users/1f1b7263ee6e467b942bcf534c7f8f99 => generated 283 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:33.338684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:33.371684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:33.424233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:33.435784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5d38a-e40e-486b-9a70-fc26e4a3e34d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:33.493642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:33.494289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:33.494576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:33.494853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:33.496031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:33.497554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:33.499561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 132/265] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:33 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:33.505986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:33.622011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e91b67-1d93-4cf4-9698-3d55b1121347 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:33.694099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:33.694760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:33.694985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:33.695243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:33.715698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ffcce-6c3a-4273-86b2-4118a090c09f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:33.800669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:33.801640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:33.802124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:33.802670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:33.803650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=1f1b7263ee6e467b942bcf534c7f8f99) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:33.804205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:33.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:33.811989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:33.817759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 133/266] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:33 2022] GET /identity/v3/role_assignments?user.id=1f1b7263ee6e467b942bcf534c7f8f99&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:33.826017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:33.827531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:33.882584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:33.977029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 134/267] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:50:33.982979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:34.154241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f78e0-2b95-4ae3-9b5e-1770ecbd75af None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:34.208740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83794f5-b3d3-4c73-921d-892b07a4132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:34.209394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83794f5-b3d3-4c73-921d-892b07a4132b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.209672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83794f5-b3d3-4c73-921d-892b07a4132b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.209945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83794f5-b3d3-4c73-921d-892b07a4132b None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.211100 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83794f5-b3d3-4c73-921d-892b07a4132b None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:34.216745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83794f5-b3d3-4c73-921d-892b07a4132b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:34.218503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 134/268] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:33 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:34.224841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:34.235823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85502b-f056-493e-a572-482c5c3c7023 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.236116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85502b-f056-493e-a572-482c5c3c7023 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.236962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85502b-f056-493e-a572-482c5c3c7023 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.238487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 135/269] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:34.243963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10e3f-fb34-407e-a996-b0e8933d3bda None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.244249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10e3f-fb34-407e-a996-b0e8933d3bda None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.244523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10e3f-fb34-407e-a996-b0e8933d3bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.272693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b3f10e3f-fb34-407e-a996-b0e8933d3bda None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:34.275116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b3f10e3f-fb34-407e-a996-b0e8933d3bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:34.411581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:34.419232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554166-43ca-424a-a0f0-db52edb83719 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:34.422681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 136/270] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:34.460642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:34.466611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0b7be-58dd-4d87-b51f-69c10a597bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:34.467308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b7be-58dd-4d87-b51f-69c10a597bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.467607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b7be-58dd-4d87-b51f-69c10a597bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.467943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b7be-58dd-4d87-b51f-69c10a597bb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.468751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0b7be-58dd-4d87-b51f-69c10a597bb2 None admin] RBAC: Adding query filter params (role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=1f1b7263ee6e467b942bcf534c7f8f99) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:34.469072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0b7be-58dd-4d87-b51f-69c10a597bb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:34.470563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0b7be-58dd-4d87-b51f-69c10a597bb2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:34.477604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 135/271] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:34 2022] GET /identity/v3/role_assignments?user.id=1f1b7263ee6e467b942bcf534c7f8f99&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=2372f2ee163c430ca4b943c833677192 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:34.495741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e583-8040-4ea6-a670-13f1f484039b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.496098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e583-8040-4ea6-a670-13f1f484039b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.496331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e583-8040-4ea6-a670-13f1f484039b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.497966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 137/272] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:34.504323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c76fc-ebe1-47d5-87fc-ea8df53decde None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.504627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c76fc-ebe1-47d5-87fc-ea8df53decde None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.504880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c76fc-ebe1-47d5-87fc-ea8df53decde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.528421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7c4c76fc-ebe1-47d5-87fc-ea8df53decde None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:34.529818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7c4c76fc-ebe1-47d5-87fc-ea8df53decde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:34.547727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3988-e212-47c9-9662-b7e014e3fa36 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.548058 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3988-e212-47c9-9662-b7e014e3fa36 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.548304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3988-e212-47c9-9662-b7e014e3fa36 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.550822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 138/273] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:34.556343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5564-c8fd-4adb-9eec-33dd638207d7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.556647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5564-c8fd-4adb-9eec-33dd638207d7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.556894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5564-c8fd-4adb-9eec-33dd638207d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.582040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-713d5564-c8fd-4adb-9eec-33dd638207d7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:34.583586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-713d5564-c8fd-4adb-9eec-33dd638207d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:34.698430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 139/274] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:34.710144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 136/275] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:34.788159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d5564-c8fd-4adb-9eec-33dd638207d7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:34.894913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d5564-c8fd-4adb-9eec-33dd638207d7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:34.923702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:34.934496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:34.935136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:34.935418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:34.935694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:34.936835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:34.938280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:34.942365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 140/276] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:34.948000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c76fc-ebe1-47d5-87fc-ea8df53decde None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:34.999668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.071878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c76fc-ebe1-47d5-87fc-ea8df53decde None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.099320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:35.109410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.110020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.110300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.110573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.111221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:35.113530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:35.117035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f02325-ebbc-4e6c-b51f-fe693809191d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.135727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 137/277] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:50:34 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:35.141215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.154791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.156359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.156359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.156803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.170089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:35.177133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:35.178383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:35.179370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:35.180987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 141/278] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:35.187098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.260389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:35.292406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be191-c407-4e66-affa-3e760bd8b0bf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.309529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bf2cd-6b40-47ba-bbb9-f0eaa23b94cc None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.332362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.333148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.333523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.333909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.348908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:35.352349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.353137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.353571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.353977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:35.354734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:35.355658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:35.356483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 138/279] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:35.357823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:35.358137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:35.359515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:35.363475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.370308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 142/280] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:35.375336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.418338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:35.496573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585712b0-7f26-4956-a67f-b023cd1cc6a8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.527141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236f626-8779-46a0-8b60-63f388f30a2c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.548380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.550163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:35.551162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.551442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.551723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.552453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:35.552746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:35.555516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:35.565763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 139/281] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:35.571726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.572266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.572913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.573196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.573471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] PATH_INFO: `/v3/users/6a56f984afc74b81900bbc4818c5780f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.591021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] RBAC: Authorizing `identity:get_user(user_id=6a56f984afc74b81900bbc4818c5780f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:35.592568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:35.599338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 143/282] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:35 2022] GET /identity/v3/users/6a56f984afc74b81900bbc4818c5780f => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:35.600955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.704032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673518b-7a60-435a-8003-d86c2c0b9caf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.707672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:35.728516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb8eed-1093-45a1-a443-d78e418641fd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:35.747585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e39ff-4d9d-44b0-8899-2d4af661801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.748412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e39ff-4d9d-44b0-8899-2d4af661801e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.748789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e39ff-4d9d-44b0-8899-2d4af661801e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.749163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e39ff-4d9d-44b0-8899-2d4af661801e None admin] PATH_INFO: `/v3/users/90fcb15b0a7145a1a4216a0ff545d0d1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.750391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e39ff-4d9d-44b0-8899-2d4af661801e None admin] RBAC: Authorizing `identity:get_user(user_id=90fcb15b0a7145a1a4216a0ff545d0d1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:35.751960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e39ff-4d9d-44b0-8899-2d4af661801e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:35.753988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 140/283] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:35 2022] GET /identity/v3/users/90fcb15b0a7145a1a4216a0ff545d0d1 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:35.757299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6af1b-d6f8-4f50-966b-8e5566cfeac5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.757299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6af1b-d6f8-4f50-966b-8e5566cfeac5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.757299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6af1b-d6f8-4f50-966b-8e5566cfeac5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.757299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 141/284] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:35.762944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3215e3-13ed-4c5c-8dbd-1bd697de9f24 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.763238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3215e3-13ed-4c5c-8dbd-1bd697de9f24 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.763607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3215e3-13ed-4c5c-8dbd-1bd697de9f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.776476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:35.777883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:35.778445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:35.778959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:35.779865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, user.id=6a56f984afc74b81900bbc4818c5780f, scope.system=all) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:35.780309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:35.782248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:35.783090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4b3215e3-13ed-4c5c-8dbd-1bd697de9f24 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:35.785055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b3215e3-13ed-4c5c-8dbd-1bd697de9f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:35.796459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 144/285] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Oct 6 10:50:35 2022] GET /identity/v3/role_assignments?user.id=6a56f984afc74b81900bbc4818c5780f&scope.system=all&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:35.799888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:35.886590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:35.927107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 142/286] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:35.986091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72fb3e-4155-43f6-bd80-bae3d0e1e8d2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:36.024422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:36.025111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:36.025397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:36.025671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:36.026801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:36.028344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:36.029921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 145/287] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:35 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:36.036949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3215e3-13ed-4c5c-8dbd-1bd697de9f24 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:36.149962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3215e3-13ed-4c5c-8dbd-1bd697de9f24 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:36.151590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:36.199211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:36.199906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:36.200190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:36.200463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25/users/90fcb15b0a7145a1a4216a0ff545d0d1/roles/6c47e35694694e9a9bbd645cae642089` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:36.205735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] RBAC: Authorizing `identity:create_grant(project_id=994f81afb7594f73a78a9d847a544f25, user_id=90fcb15b0a7145a1a4216a0ff545d0d1, role_id=6c47e35694694e9a9bbd645cae642089)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:36.207281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:36.232648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 143/288] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:50:36 2022] PUT /identity/v3/projects/994f81afb7594f73a78a9d847a544f25/users/90fcb15b0a7145a1a4216a0ff545d0d1/roles/6c47e35694694e9a9bbd645cae642089 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 10:50:36.379094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:36.515757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1d302-d6e3-4d83-849b-a9e15ddc36a8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:36.526864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:36.557432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:36.558078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:36.558361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:36.558635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:36.571330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:36.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:36.577698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:36.578678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 146/289] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:36.584334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:36.736114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e1094-2e09-46a0-9655-94ebf3b7b56f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:36.780610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:36.781234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:36.781749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:36.782026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:36.782694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:36.782995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:36.784966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:36.799624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 144/290] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:36.804944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:36.946984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda5d1e-3e1c-47de-befb-6fd1e283de66 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:36.990978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:36.991963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:36.992453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:36.992912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] PATH_INFO: `/v3/users/ae5aad576afe4003b431336da0242306` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:36.994247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] RBAC: Authorizing `identity:get_user(user_id=ae5aad576afe4003b431336da0242306)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:36.995944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:36.998057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 147/291] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:36 2022] GET /identity/v3/users/ae5aad576afe4003b431336da0242306 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:37.003987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:37.053211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:37.155585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d663c-253a-4932-beb0-29de64c0c72c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:37.203323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:37.203972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:37.204258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:37.204535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:37.299384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:37.300796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:37.339192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:37.458727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 145/292] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:37.464594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:37.528684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:37.601761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251486d-f22b-4e33-924c-a9d9c40146f1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:37.637547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb18295-2e03-4626-a892-3dd6a32df66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:37.638411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb18295-2e03-4626-a892-3dd6a32df66f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:37.638863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb18295-2e03-4626-a892-3dd6a32df66f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:37.639269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb18295-2e03-4626-a892-3dd6a32df66f None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:37.640565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb18295-2e03-4626-a892-3dd6a32df66f None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:37.642160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb18295-2e03-4626-a892-3dd6a32df66f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:37.643999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 148/293] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:37 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:37.652211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:37.791355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27debb-9d62-48b9-a288-32dc67c9ec10 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:37.821444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:37.832587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8361c4-f56a-4121-b37d-d0ea47b5484e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:37.833455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8361c4-f56a-4121-b37d-d0ea47b5484e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:37.834071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8361c4-f56a-4121-b37d-d0ea47b5484e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:37.834339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8361c4-f56a-4121-b37d-d0ea47b5484e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:37.835067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8361c4-f56a-4121-b37d-d0ea47b5484e None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=ae5aad576afe4003b431336da0242306) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:37.835373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8361c4-f56a-4121-b37d-d0ea47b5484e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:37.837002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8361c4-f56a-4121-b37d-d0ea47b5484e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:37.843353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 146/294] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:37 2022] GET /identity/v3/role_assignments?user.id=ae5aad576afe4003b431336da0242306&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:38.023864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550c241-9771-45c2-aadf-6d9eb67a2d45 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.024940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550c241-9771-45c2-aadf-6d9eb67a2d45 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.025417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550c241-9771-45c2-aadf-6d9eb67a2d45 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.027361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 149/295] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:38.034214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeaf5b-ba9e-46a2-8161-6062611368ad None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.034775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeaf5b-ba9e-46a2-8161-6062611368ad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.035265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeaf5b-ba9e-46a2-8161-6062611368ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.055289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b4eeaf5b-ba9e-46a2-8161-6062611368ad None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:38.056887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b4eeaf5b-ba9e-46a2-8161-6062611368ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:38.139762 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 147/296] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:38.212890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904aacf-7bd3-4120-94c9-05fea31f972e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.213422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904aacf-7bd3-4120-94c9-05fea31f972e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.213837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904aacf-7bd3-4120-94c9-05fea31f972e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.215302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 150/297] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:38.221948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc639c4e-0447-4aeb-be41-653b577ce75d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.222408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc639c4e-0447-4aeb-be41-653b577ce75d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.222821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc639c4e-0447-4aeb-be41-653b577ce75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.242700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dc639c4e-0447-4aeb-be41-653b577ce75d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:38.244128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc639c4e-0447-4aeb-be41-653b577ce75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:38.320005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 148/298] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:38.453267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904aacf-7bd3-4120-94c9-05fea31f972e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:38.521528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e85a6-ee8c-4130-a66a-094135a3f403 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.522143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e85a6-ee8c-4130-a66a-094135a3f403 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.522614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e85a6-ee8c-4130-a66a-094135a3f403 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.524185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 149/299] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:50:38.529214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac20a-ef19-48f1-ba0b-63d6933e48f2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.529489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac20a-ef19-48f1-ba0b-63d6933e48f2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.529760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bac20a-ef19-48f1-ba0b-63d6933e48f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.558056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-93bac20a-ef19-48f1-ba0b-63d6933e48f2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:38.559230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93bac20a-ef19-48f1-ba0b-63d6933e48f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:38.579669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904aacf-7bd3-4120-94c9-05fea31f972e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:38.582800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:38.620971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:38.621615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:38.622326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.622829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.623104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.637610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:38.642421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:38.644458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 151/300] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:38.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:38.655897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 150/301] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:38.655897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bac20a-ef19-48f1-ba0b-63d6933e48f2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:38.762361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bac20a-ef19-48f1-ba0b-63d6933e48f2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:38.780749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe05bdd-8492-43af-971b-e1c640677cc6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:38.798474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:38.799694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.799937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.800202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.800881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:38.801164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:38.802531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:38.812503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 151/302] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:38.818422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:38.825993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:38.826803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.827245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.827653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.829430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:38.831134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:38.833755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 152/303] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:38.839043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:38.872507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:38.880715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:38.954184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bf0b8-d1e3-4378-9b33-369f6b3b129f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:38.961445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24da10-14ff-4501-8389-5e205875a7d2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:38.991237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:38.991862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.992164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.992436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] PATH_INFO: `/v3/groups` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:38.993064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:38.994793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:38.998412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf6378-9e96-4d99-85fc-483614273c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:38.999126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf6378-9e96-4d99-85fc-483614273c06 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:38.999413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf6378-9e96-4d99-85fc-483614273c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:38.999690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf6378-9e96-4d99-85fc-483614273c06 None admin] PATH_INFO: `/v3/users/c5bb754ccfea4c2eb91f5fef843d74f2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.011209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 153/304] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Oct 6 10:50:38 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:50:39.012064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:39.020008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf6378-9e96-4d99-85fc-483614273c06 None admin] RBAC: Authorizing `identity:get_user(user_id=c5bb754ccfea4c2eb91f5fef843d74f2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:39.021486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf6378-9e96-4d99-85fc-483614273c06 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:39.023376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 152/305] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:38 2022] GET /identity/v3/users/c5bb754ccfea4c2eb91f5fef843d74f2 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:39.026435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18643a1e-3f1a-4528-8207-92fcc07d9652 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.026435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18643a1e-3f1a-4528-8207-92fcc07d9652 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.026435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18643a1e-3f1a-4528-8207-92fcc07d9652 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.026754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 153/306] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:39.030144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18643a1e-3f1a-4528-8207-92fcc07d9652 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:39.129942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:39.139598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2e4b6-0200-4ee9-9cd1-a27595150daf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:39.146653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:39.152187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18643a1e-3f1a-4528-8207-92fcc07d9652 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:39.179466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7ccc1-a9b7-4ec3-bd00-bf0d459dc07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:39.180148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ccc1-a9b7-4ec3-bd00-bf0d459dc07e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.180446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ccc1-a9b7-4ec3-bd00-bf0d459dc07e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.180715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ccc1-a9b7-4ec3-bd00-bf0d459dc07e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.192352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7ccc1-a9b7-4ec3-bd00-bf0d459dc07e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:39.195259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:39.195908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.196195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.196471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.197203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] RBAC: Adding query filter params (role.id=6c47e35694694e9a9bbd645cae642089, user.id=c5bb754ccfea4c2eb91f5fef843d74f2, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:39.197513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c5c7ccc1-a9b7-4ec3-bd00-bf0d459dc07e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:39.198413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:39.199253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:39.200079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:39.200710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:39.200777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 154/307] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:39.201357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1271e44-e32f-43f4-ab23-ba1c02abadec None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.201357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1271e44-e32f-43f4-ab23-ba1c02abadec None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.201582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1271e44-e32f-43f4-ab23-ba1c02abadec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.212132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 154/308] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Oct 6 10:50:39 2022] GET /identity/v3/role_assignments?user.id=c5bb754ccfea4c2eb91f5fef843d74f2&scope.system=all&role.id=6c47e35694694e9a9bbd645cae642089 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:39.212933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:39.229060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c1271e44-e32f-43f4-ab23-ba1c02abadec None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:39.230289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c1271e44-e32f-43f4-ab23-ba1c02abadec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:39.314630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 155/309] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:39.347437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fa238-51a6-4424-ac24-b264da59a17e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:39.392152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:39.393701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.394010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.394285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.394976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:39.395280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:39.402082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:39.413343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 155/310] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:39.419006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1271e44-e32f-43f4-ab23-ba1c02abadec None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:39.507538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:39.528557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1271e44-e32f-43f4-ab23-ba1c02abadec None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:39.566501 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:39.567135 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.567412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.567685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] PATH_INFO: `/v3/users/6a56f984afc74b81900bbc4818c5780f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.568840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] RBAC: Authorizing `identity:get_user(user_id=6a56f984afc74b81900bbc4818c5780f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:39.570376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:39.578472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 156/311] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:39 2022] GET /identity/v3/users/6a56f984afc74b81900bbc4818c5780f => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:39.584672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:39.610842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:39.620744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:39.697744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786cf7d-6935-497b-b270-992b0ddf0fdf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:39.733355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:39.734020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.734305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.734585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] PATH_INFO: `/v3/system/users/6a56f984afc74b81900bbc4818c5780f/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.736044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a56f984afc74b81900bbc4818c5780f, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:39.738364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:39.744102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dd2ae-2905-40a7-8ed4-6e7b1ca45294 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:39.758026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 156/312] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Oct 6 10:50:39 2022] PUT /identity/v3/system/users/6a56f984afc74b81900bbc4818c5780f/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:39.791318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:39.792297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.792722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.793171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.806631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:39.812757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:39.813929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:39.815335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:39.817933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 157/313] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:39.824252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:39.927240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:39.944887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63547ce-4b26-4cac-a1dc-e918424cecfb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:39.981208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:39.982047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:39.982487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:39.982916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:39.983730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:39.984185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:39.985699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:39.995881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 157/314] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:40.003955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:40.017390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:40.119152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a35a0-f9da-4855-be63-20f5e7769b92 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:40.155454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:40.157106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:40.157589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:40.158061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] PATH_INFO: `/v3/users/90fcb15b0a7145a1a4216a0ff545d0d1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:40.159376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] RBAC: Authorizing `identity:get_user(user_id=90fcb15b0a7145a1a4216a0ff545d0d1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:40.161022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:40.163093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 158/315] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:40 2022] GET /identity/v3/users/90fcb15b0a7145a1a4216a0ff545d0d1 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:40.168382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:40.278745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a0300-d7ca-4391-9720-75d6db50810b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:40.312941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:40.313602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:40.313885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:40.314154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:40.334086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:40.389800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:40.391223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:40.493410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 158/316] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:40.499008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:40.583984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:40.608082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ca6a7-cad7-45f5-99cb-dd9d8795e952 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:40.643374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb44c9a-1e43-4e59-b48e-9c5957c4976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:40.644037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44c9a-1e43-4e59-b48e-9c5957c4976e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:40.644321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44c9a-1e43-4e59-b48e-9c5957c4976e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:40.644597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44c9a-1e43-4e59-b48e-9c5957c4976e None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:40.645730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb44c9a-1e43-4e59-b48e-9c5957c4976e None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:40.647186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb44c9a-1e43-4e59-b48e-9c5957c4976e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:40.648853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 159/317] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:40 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:40.654747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:40.708685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:40.759767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609bd2f-3d81-4896-870e-69657cdac8d8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:40.795634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53546df-c08b-43f5-9b39-aac841e2c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:40.796278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53546df-c08b-43f5-9b39-aac841e2c06b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:40.796559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53546df-c08b-43f5-9b39-aac841e2c06b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:40.796835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53546df-c08b-43f5-9b39-aac841e2c06b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:40.797566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53546df-c08b-43f5-9b39-aac841e2c06b None admin] RBAC: Adding query filter params (role.id=6c47e35694694e9a9bbd645cae642089, scope.project.id=994f81afb7594f73a78a9d847a544f25, user.id=90fcb15b0a7145a1a4216a0ff545d0d1) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:40.797874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53546df-c08b-43f5-9b39-aac841e2c06b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:40.799353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53546df-c08b-43f5-9b39-aac841e2c06b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:40.805563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 159/318] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:50:40 2022] GET /identity/v3/role_assignments?user.id=90fcb15b0a7145a1a4216a0ff545d0d1&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=6c47e35694694e9a9bbd645cae642089 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:41.333052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b82cfb-4b42-4bb3-99da-54cf0d3ab326 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:41.333589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b82cfb-4b42-4bb3-99da-54cf0d3ab326 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:41.333589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b82cfb-4b42-4bb3-99da-54cf0d3ab326 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:41.334921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 160/319] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:41.340166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531c16-19bc-4ce1-872a-1210cc5cc8f1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:41.340561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531c16-19bc-4ce1-872a-1210cc5cc8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:41.340704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531c16-19bc-4ce1-872a-1210cc5cc8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:41.359215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-40531c16-19bc-4ce1-872a-1210cc5cc8f1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:41.360400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-40531c16-19bc-4ce1-872a-1210cc5cc8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:41.443870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 160/320] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:41.516246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d56e05-8cf3-428e-84c8-20e07a5e1f9d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:41.516628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d56e05-8cf3-428e-84c8-20e07a5e1f9d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:41.516971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d56e05-8cf3-428e-84c8-20e07a5e1f9d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:41.519027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 161/321] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:41.524406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e7c2-dc41-4eb3-82df-46b90242fbe7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:41.524698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e7c2-dc41-4eb3-82df-46b90242fbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:41.524974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e7c2-dc41-4eb3-82df-46b90242fbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:41.544109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3a31e7c2-dc41-4eb3-82df-46b90242fbe7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:41.545296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3a31e7c2-dc41-4eb3-82df-46b90242fbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:41.587999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:41.616283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 161/322] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:41.712825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d56e05-8cf3-428e-84c8-20e07a5e1f9d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:41.816645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d56e05-8cf3-428e-84c8-20e07a5e1f9d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:41.853791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:41.854342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:41.854619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:41.854894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:41.856020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:41.857474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:41.859235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 162/323] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:50:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:41.864992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e7c2-dc41-4eb3-82df-46b90242fbe7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:41.887558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:41.906594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:41.979141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e7c2-dc41-4eb3-82df-46b90242fbe7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.002386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504dd47-4823-400c-9e99-4c058ffaa4f2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.017189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dff9a-7b45-4bd4-a6c6-4a7e3a150f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.018557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dff9a-7b45-4bd4-a6c6-4a7e3a150f4e None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.018862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dff9a-7b45-4bd4-a6c6-4a7e3a150f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.019120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dff9a-7b45-4bd4-a6c6-4a7e3a150f4e None admin] PATH_INFO: `/v3/groups` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.019767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dff9a-7b45-4bd4-a6c6-4a7e3a150f4e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:42.021248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dff9a-7b45-4bd4-a6c6-4a7e3a150f4e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:42.038502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 162/324] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Oct 6 10:50:41 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:50:42.039778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123df1e2-c618-4731-8028-c7a7d8815640 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.040105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123df1e2-c618-4731-8028-c7a7d8815640 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.040385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123df1e2-c618-4731-8028-c7a7d8815640 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.040725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.041610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 163/325] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:42.041722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.042110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.042571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.047325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10890d43-a6ed-483e-acac-02ee0132771b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.047614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10890d43-a6ed-483e-acac-02ee0132771b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.047938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10890d43-a6ed-483e-acac-02ee0132771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.057996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:42.063644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:42.064743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:42.065691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 163/326] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:42.071054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:42.072798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-10890d43-a6ed-483e-acac-02ee0132771b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:42.074589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-10890d43-a6ed-483e-acac-02ee0132771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:42.114223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:42.159374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:42.165447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 164/327] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:42.188009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119498d6-6d75-4c26-b644-39425a996185 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.227223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.227223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.227223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.227579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.228566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:42.228566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:42.229873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:42.238910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 164/328] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:42.244467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10890d43-a6ed-483e-acac-02ee0132771b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:42.356456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10890d43-a6ed-483e-acac-02ee0132771b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.397435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.398097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.398381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.398654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] PATH_INFO: `/v3/users/c5bb754ccfea4c2eb91f5fef843d74f2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.399809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] RBAC: Authorizing `identity:get_user(user_id=c5bb754ccfea4c2eb91f5fef843d74f2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:42.401280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:42.406978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 165/329] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:42 2022] GET /identity/v3/users/c5bb754ccfea4c2eb91f5fef843d74f2 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:42.409173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:42.438238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:42.478450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:42.530140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9d4a-80ad-48ea-b1f5-31d74fe02720 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.568491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3cab9-1e71-4c6f-94cb-fa84f33e6560 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.570357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.571019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.571301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.571580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] PATH_INFO: `/v3/system/users/c5bb754ccfea4c2eb91f5fef843d74f2/roles/6c47e35694694e9a9bbd645cae642089` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.573044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c5bb754ccfea4c2eb91f5fef843d74f2, role_id=6c47e35694694e9a9bbd645cae642089)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:42.574417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:42.578113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:42.587240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 165/330] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Oct 6 10:50:42 2022] PUT /identity/v3/system/users/c5bb754ccfea4c2eb91f5fef843d74f2/roles/6c47e35694694e9a9bbd645cae642089 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:42.623743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.624443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.624722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.624999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.640886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:42.646282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:42.647289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:42.648128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 166/331] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:42.653694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:42.778940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241d9d6-ac31-4ac6-b080-ef7f4263fdad None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.814533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.815140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.815415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.815684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:42.816383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:42.816692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:42.818076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:42.826934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 166/332] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:42.833315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:42.935557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:42.949905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba41a-3e14-4097-9ea1-9016227c77a4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:42.997119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c041666-3613-4a66-80b0-d7f9d1a2fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:42.997770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c041666-3613-4a66-80b0-d7f9d1a2fd4c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:42.998050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c041666-3613-4a66-80b0-d7f9d1a2fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:42.998322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c041666-3613-4a66-80b0-d7f9d1a2fd4c None admin] PATH_INFO: `/v3/users/6a56f984afc74b81900bbc4818c5780f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:43.000093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c041666-3613-4a66-80b0-d7f9d1a2fd4c None admin] RBAC: Authorizing `identity:get_user(user_id=6a56f984afc74b81900bbc4818c5780f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:43.001540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c041666-3613-4a66-80b0-d7f9d1a2fd4c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:43.004096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 167/333] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:42 2022] GET /identity/v3/users/6a56f984afc74b81900bbc4818c5780f => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:43.009564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:43.088417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:43.133615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c54a6-c14d-4de1-9794-ddac0fd2ead4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:43.175732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1fd25-e9fd-46e9-947a-9d33a4100345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:43.176411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1fd25-e9fd-46e9-947a-9d33a4100345 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:43.176691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1fd25-e9fd-46e9-947a-9d33a4100345 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:43.177012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1fd25-e9fd-46e9-947a-9d33a4100345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:43.177757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1fd25-e9fd-46e9-947a-9d33a4100345 None admin] RBAC: Adding query filter params (role.id=1a3c6cbe3fb44e9a812652129507717e, user.id=6a56f984afc74b81900bbc4818c5780f, scope.system=all) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:43.178065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1fd25-e9fd-46e9-947a-9d33a4100345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:43.179690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1fd25-e9fd-46e9-947a-9d33a4100345 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:43.200102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 167/334] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Oct 6 10:50:43 2022] GET /identity/v3/role_assignments?user.id=6a56f984afc74b81900bbc4818c5780f&scope.system=all&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:44.624927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45953dd-c563-44c8-a672-a36f27a985b9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:44.625433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45953dd-c563-44c8-a672-a36f27a985b9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:44.625537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45953dd-c563-44c8-a672-a36f27a985b9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:44.627337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 168/335] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:44.633309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a5a1d-8171-49a2-b4f4-40170a1d69fe None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:44.634169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a5a1d-8171-49a2-b4f4-40170a1d69fe None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:44.634659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a5a1d-8171-49a2-b4f4-40170a1d69fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:44.659222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dc9a5a1d-8171-49a2-b4f4-40170a1d69fe None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:44.660759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dc9a5a1d-8171-49a2-b4f4-40170a1d69fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:44.741169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 168/336] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:44.849751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0629-aa61-4f24-8632-86655e7c4af4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:44.850380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0629-aa61-4f24-8632-86655e7c4af4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:44.850881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0629-aa61-4f24-8632-86655e7c4af4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:44.852448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 169/337] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:44.858044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55802-7ccc-4320-8811-bf4f40b26595 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:44.858582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55802-7ccc-4320-8811-bf4f40b26595 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:44.859116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55802-7ccc-4320-8811-bf4f40b26595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:44.882813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c2d55802-7ccc-4320-8811-bf4f40b26595 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:44.884369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c2d55802-7ccc-4320-8811-bf4f40b26595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:44.935911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:44.961556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 169/338] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:45.178679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad0629-aa61-4f24-8632-86655e7c4af4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:45.300977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad0629-aa61-4f24-8632-86655e7c4af4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:45.343713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:45.344382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:45.344671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:45.345106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:45.359150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:45.364692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:45.365568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:45.366388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 170/339] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:45.371749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d55802-7ccc-4320-8811-bf4f40b26595 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:45.449782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:45.477182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:45.510021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d55802-7ccc-4320-8811-bf4f40b26595 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:45.573813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:45.574753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:45.575251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:45.575733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:45.576685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:45.577407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:45.581290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:45.600127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 170/340] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:45.605994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:45.627407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaec43-17e2-4ba0-9a15-77a495036dc2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:45.688531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:45.693096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:45.693747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:45.694027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:45.694356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:45.724756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:45.729652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:45.730530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:45.731321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 171/341] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:45.738998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:45.776517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff586d8-77e5-44b8-8b5d-d5816164c66a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:45.821398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:45.822063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:45.822343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:45.822902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:45.894905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:45.909347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:45.911056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:45.925127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbe8f0-aa10-4752-80d1-7a18d165c0f4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:45.993798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:45.997738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:45.998356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:45.998612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.000350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.001025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:46.001324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:46.003485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:46.024073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 172/342] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:46.035196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:46.143626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 171/343] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:46.160210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:46.259792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513de03-a0a4-4f86-95fb-41db4373eb2a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:46.321882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f5130-e55c-481c-bcfa-83d29ae9c70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:46.322776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f5130-e55c-481c-bcfa-83d29ae9c70b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.323202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f5130-e55c-481c-bcfa-83d29ae9c70b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.323771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f5130-e55c-481c-bcfa-83d29ae9c70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.334952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b92fb-3430-4de6-845b-720b32835dc6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:46.339955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:46.373557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b500b-3d7d-4512-9339-ecfadf33438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:46.374468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b500b-3d7d-4512-9339-ecfadf33438e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.374929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b500b-3d7d-4512-9339-ecfadf33438e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.375374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b500b-3d7d-4512-9339-ecfadf33438e None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.378143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b500b-3d7d-4512-9339-ecfadf33438e None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:46.379820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b500b-3d7d-4512-9339-ecfadf33438e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:46.381985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 172/344] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:46 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:46.383322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575de4e-df9a-48c9-b06f-2517da146e3b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.383778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575de4e-df9a-48c9-b06f-2517da146e3b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.384248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575de4e-df9a-48c9-b06f-2517da146e3b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.385618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 173/345] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:46.386853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961680d-3ee5-42e5-a4a3-96542d0f9daf None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.387371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961680d-3ee5-42e5-a4a3-96542d0f9daf None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.387804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961680d-3ee5-42e5-a4a3-96542d0f9daf None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.389465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 174/346] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:46.397705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e2be2-f043-4147-8286-43f7e52fce10 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.398297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e2be2-f043-4147-8286-43f7e52fce10 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.398772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e2be2-f043-4147-8286-43f7e52fce10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.418284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f5130-e55c-481c-bcfa-83d29ae9c70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:46.420286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f5130-e55c-481c-bcfa-83d29ae9c70b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:46.420507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e81e2be2-f043-4147-8286-43f7e52fce10 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:46.421740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e81e2be2-f043-4147-8286-43f7e52fce10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:46.426867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:46.519341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 175/347] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:46.520232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e2be2-f043-4147-8286-43f7e52fce10 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:46.557139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 173/348] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:46.558544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675fc5-566e-4ab9-a363-bf2f162c8915 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.558883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675fc5-566e-4ab9-a363-bf2f162c8915 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.559135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675fc5-566e-4ab9-a363-bf2f162c8915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.583682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-10675fc5-566e-4ab9-a363-bf2f162c8915 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:46.585184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-10675fc5-566e-4ab9-a363-bf2f162c8915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:46.623599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:46.660053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 174/349] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:46.660763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675fc5-566e-4ab9-a363-bf2f162c8915 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:46.732466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e2be2-f043-4147-8286-43f7e52fce10 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:46.772293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675fc5-566e-4ab9-a363-bf2f162c8915 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:46.809977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:46.810890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.811406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.811816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.812304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:46.813042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.813422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.813609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:46.813926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.816327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:46.818140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 175/350] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:46 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:46.823598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:46.844545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:46.846937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:46.848662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 176/351] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:46 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:46.854546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:46.870474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:46.873093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:46.938365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c4ff2-8503-4551-bfdb-4c5e78d81074 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:46.979707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:46.980370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:46.980701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:46.980979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:46.982084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:46.983732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:46.985915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 176/352] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:46 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:46.986686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:47.055241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f526b7-c102-4b96-8c0a-fd42ac7f0232 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:47.105542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:47.107275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.107707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.108171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.109141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=7f87007362fc43378dacfe7a12000392) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:47.109592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:47.111295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:47.119202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 177/353] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:46 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:47.120420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:47.144550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d46aae-c8d6-4f72-9388-dc58e400ea4e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:47.156510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:47.194324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c0589-c229-4c3f-9167-e86ec73c26de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:47.195282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0589-c229-4c3f-9167-e86ec73c26de None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.195770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0589-c229-4c3f-9167-e86ec73c26de None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.196287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0589-c229-4c3f-9167-e86ec73c26de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.210431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c0589-c229-4c3f-9167-e86ec73c26de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f45c0589-c229-4c3f-9167-e86ec73c26de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:47.216836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:47.217808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 10:50:47.218628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:47.221002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:47.221405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 177/354] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:47.223025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df2b0-af30-463d-8ac5-1e247d0694a2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.223584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df2b0-af30-463d-8ac5-1e247d0694a2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.224151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df2b0-af30-463d-8ac5-1e247d0694a2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.225643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 178/355] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:47.226685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9df2b0-af30-463d-8ac5-1e247d0694a2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:47.284171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92539ae5-e93d-4561-bd1c-551b19ea332f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:47.341402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:47.341402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.341402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.341740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.342272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=7f87007362fc43378dacfe7a12000392) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:47.342578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:47.344275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:47.352421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 178/356] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:47 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=2372f2ee163c430ca4b943c833677192 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:47.353261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:47.367856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9df2b0-af30-463d-8ac5-1e247d0694a2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:47.425628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11298ad-9c2e-4d90-8912-bf74ef442479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:47.426287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11298ad-9c2e-4d90-8912-bf74ef442479 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.426607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11298ad-9c2e-4d90-8912-bf74ef442479 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.426887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11298ad-9c2e-4d90-8912-bf74ef442479 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.444649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11298ad-9c2e-4d90-8912-bf74ef442479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b11298ad-9c2e-4d90-8912-bf74ef442479 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:47.451498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:47.452470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:47.453263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 179/357] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:47.455679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1de3e-fb75-4689-9c6b-95898f03493b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.459268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1de3e-fb75-4689-9c6b-95898f03493b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.459268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1de3e-fb75-4689-9c6b-95898f03493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.493091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-85c1de3e-fb75-4689-9c6b-95898f03493b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:47.494797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-85c1de3e-fb75-4689-9c6b-95898f03493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:47.542298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:47.548852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02239-4d99-465a-b9d3-236e3e53be84 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:47.600631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:47.609144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:47.610442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.612099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.612602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.613703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:47.615663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:47.617315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:47.629647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 179/358] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:47.630798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:47.657964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 180/359] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:47.659429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39c7fc-54e2-4e36-86d2-71065c1c59b6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.659713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39c7fc-54e2-4e36-86d2-71065c1c59b6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.660011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39c7fc-54e2-4e36-86d2-71065c1c59b6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.661400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 181/360] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:47.661915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39c7fc-54e2-4e36-86d2-71065c1c59b6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:47.825848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aabd-453f-468a-98b7-92b280882cf1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:47.837035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39c7fc-54e2-4e36-86d2-71065c1c59b6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:47.892817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:47.893436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.893715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.893983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.894649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:47.894949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:47.896345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:47.900815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b77ef-7183-4e0c-b7ee-1568a1a0be85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:47.901445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b77ef-7183-4e0c-b7ee-1568a1a0be85 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b77ef-7183-4e0c-b7ee-1568a1a0be85 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.901988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b77ef-7183-4e0c-b7ee-1568a1a0be85 None admin] PATH_INFO: `/v3/users/c5bb754ccfea4c2eb91f5fef843d74f2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.903744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b77ef-7183-4e0c-b7ee-1568a1a0be85 None admin] RBAC: Authorizing `identity:get_user(user_id=c5bb754ccfea4c2eb91f5fef843d74f2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:47.905182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b77ef-7183-4e0c-b7ee-1568a1a0be85 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:47.907511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 182/361] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:50:47 2022] GET /identity/v3/users/c5bb754ccfea4c2eb91f5fef843d74f2 => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:47.912700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3710d-c576-4f64-9bd9-e2312bd18104 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:47.912982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3710d-c576-4f64-9bd9-e2312bd18104 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:47.913258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3710d-c576-4f64-9bd9-e2312bd18104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:47.918977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 180/362] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:47.919831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:47.950377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b9e3710d-c576-4f64-9bd9-e2312bd18104 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:47.951577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b9e3710d-c576-4f64-9bd9-e2312bd18104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:47.983871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:48.016533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:48.044264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 183/363] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:48.045139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3710d-c576-4f64-9bd9-e2312bd18104 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:48.077088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2880bf6-e57d-42e2-b795-65b966b79a59 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.118769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:48.119376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:48.119652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:48.119947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:48.120911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] RBAC: Adding query filter params (role.id=6c47e35694694e9a9bbd645cae642089, user.id=c5bb754ccfea4c2eb91f5fef843d74f2, scope.system=all) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:48.121221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:48.122647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:48.136303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 181/364] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Oct 6 10:50:47 2022] GET /identity/v3/role_assignments?user.id=c5bb754ccfea4c2eb91f5fef843d74f2&scope.system=all&role.id=6c47e35694694e9a9bbd645cae642089 => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:48.137244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:48.165484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3710d-c576-4f64-9bd9-e2312bd18104 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.210977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:48.211598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:48.211908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:48.212182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:48.240555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:48.263082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec09e-dde3-4912-9be5-1338516f07bf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.278750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:48.290504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:48.292546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:48.308723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:48.309376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:48.309654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:48.309925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:48.335448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:48.343451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:48.344571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:48.347274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 182/365] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:48.353438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:48.411535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 184/366] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:48.420037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:48.482519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:48.534567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77a219-021f-4daf-816c-d0d0aefddb20 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.539571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2d77c-d008-4881-89f3-f00b6b173eb9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.585168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:48.586182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:48.586614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:48.587046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:48.588361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:48.589943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:48.593529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 185/367] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:48 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:48.601007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:48.607860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:48.607860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:48.607860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:48.607860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:48.608258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:48.608258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:48.608258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:48.618541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 183/368] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:48.625579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:48.675463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:48.758519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bade6f-2977-47e3-a1df-a0a765ec6177 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.765785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c7e7c-3692-400c-8b37-33da04989e23 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.792605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:48.812930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:48.814623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:48.815212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:48.816775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:48.818739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:48.820958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:48.823159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 186/369] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:48 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:48.824255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:48.833688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d023-8596-4a15-840d-217447c5220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:48.834365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d023-8596-4a15-840d-217447c5220f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:48.834694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d023-8596-4a15-840d-217447c5220f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:48.834970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d023-8596-4a15-840d-217447c5220f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:48.933921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079d023-8596-4a15-840d-217447c5220f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:48.936236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079d023-8596-4a15-840d-217447c5220f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:48.969685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a92669-fcc2-4b58-9b47-9bb4f1170bd9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:48.994150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:49.025385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:49.026025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:49.026311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:49.026578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:49.042573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:49.049763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:49.050659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:49.051461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:49.053221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 187/370] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:49.053849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:49.099851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 184/371] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:50:49.101073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d023-8596-4a15-840d-217447c5220f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:49.124818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:49.200513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef844ff-32f4-4fad-940b-880320ed71ed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:49.246839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d023-8596-4a15-840d-217447c5220f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:49.261246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:49.261849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:49.262138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:49.262415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:49.263150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=994f81afb7594f73a78a9d847a544f25) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:49.263461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:49.265081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:49.275782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 188/372] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:49 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:49.276717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:49.304703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:49.305687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:49.306180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:49.306648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:49.307551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:49.308072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:49.309642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:49.317381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:49.328855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 185/373] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:49.334954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:49.445149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487df345-b409-4e1e-ae45-b952f50d001f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:49.483223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:49.512394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53454dcf-d959-4444-be8b-ea996d2df2fb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:49.519712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:49.528641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:49.529181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:49.529622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:49.531361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:49.533429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:49.535139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 189/374] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:49 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:49.544909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:49.561882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:49.562921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:49.563404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:49.563984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:49.675189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:49.679594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:49.681390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:49.710120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558ab0b-95b1-4b90-9176-b7c8c083042d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:49.772733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:49.774185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:49.776148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:49.776688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:49.778866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:49.780575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:49.784147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 190/375] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:49 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:49.794617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:49.884868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 186/376] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:49.890780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:49.924974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:50.000905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:50.005818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a77a9-a0c0-41ea-9c67-5843538968f0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:50.027040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12b75f-1795-42f1-94a7-b007d8bdc3db None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:50.046552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bb9e7-8e77-490a-acb7-d30c850a9a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:50.047225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bb9e7-8e77-490a-acb7-d30c850a9a84 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.048027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bb9e7-8e77-490a-acb7-d30c850a9a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.048027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bb9e7-8e77-490a-acb7-d30c850a9a84 None admin] PATH_INFO: `/v3/projects/46752b92e78642e4b79f5ef50f2e51da` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.050553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bb9e7-8e77-490a-acb7-d30c850a9a84 None admin] RBAC: Authorizing `identity:get_project(project_id=46752b92e78642e4b79f5ef50f2e51da)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:50.052592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bb9e7-8e77-490a-acb7-d30c850a9a84 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:50.055874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 187/377] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:49 2022] GET /identity/v3/projects/46752b92e78642e4b79f5ef50f2e51da => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:50.055874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a72c4-88fa-4044-b797-8b8fbc436985 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.055874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a72c4-88fa-4044-b797-8b8fbc436985 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.055874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a72c4-88fa-4044-b797-8b8fbc436985 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.057350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 188/378] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:50.069885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a72c4-88fa-4044-b797-8b8fbc436985 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:50.074938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c2bd0-15a4-4682-a37e-c63f8f815b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:50.075955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c2bd0-15a4-4682-a37e-c63f8f815b0e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.076462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c2bd0-15a4-4682-a37e-c63f8f815b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.076928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c2bd0-15a4-4682-a37e-c63f8f815b0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.077853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c2bd0-15a4-4682-a37e-c63f8f815b0e None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=994f81afb7594f73a78a9d847a544f25) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:50.078365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c2bd0-15a4-4682-a37e-c63f8f815b0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:50.080184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c2bd0-15a4-4682-a37e-c63f8f815b0e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:50.088092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 191/379] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:49 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=2372f2ee163c430ca4b943c833677192 => generated 260 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:50.089669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da985b-41d0-4395-a6c7-edd4ccc4477e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.090164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da985b-41d0-4395-a6c7-edd4ccc4477e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.090676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da985b-41d0-4395-a6c7-edd4ccc4477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.115335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b6da985b-41d0-4395-a6c7-edd4ccc4477e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:50.117091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b6da985b-41d0-4395-a6c7-edd4ccc4477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:50.217968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 192/380] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:50.257099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a72c4-88fa-4044-b797-8b8fbc436985 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:50.311019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:50.311333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.311611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.311899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] PATH_INFO: `/v3/groups/d1fdb6ae834e4efdae606e0e04d6c74e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.313677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270b78-900d-4b96-9cd5-4f6d417a890b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.314293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270b78-900d-4b96-9cd5-4f6d417a890b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.314808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270b78-900d-4b96-9cd5-4f6d417a890b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.316665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 193/381] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:50.321058 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] RBAC: Authorizing `identity:get_group(group_id=d1fdb6ae834e4efdae606e0e04d6c74e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:50.322205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44032338-a051-4856-8336-507d63bd79d5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.322537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:50.322910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44032338-a051-4856-8336-507d63bd79d5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.323375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44032338-a051-4856-8336-507d63bd79d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.324277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 189/382] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:50 2022] GET /identity/v3/groups/d1fdb6ae834e4efdae606e0e04d6c74e => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:50.333960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:50.363082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-44032338-a051-4856-8336-507d63bd79d5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:50.364961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-44032338-a051-4856-8336-507d63bd79d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:50.389861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:50.428434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:50.470361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 194/383] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:50.501261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072d143-1b3a-4910-b6c0-140ec1df16da None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:50.543297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:50.543951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.544231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.544503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.545231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] RBAC: Adding query filter params (group.id=d1fdb6ae834e4efdae606e0e04d6c74e, role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=46752b92e78642e4b79f5ef50f2e51da) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:50.545536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:50.546976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:50.554187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 190/384] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:50 2022] GET /identity/v3/role_assignments?group.id=d1fdb6ae834e4efdae606e0e04d6c74e&scope.project.id=46752b92e78642e4b79f5ef50f2e51da&role.id=4c41d809cc274ff5871dc8092f05715a => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:50.619347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44032338-a051-4856-8336-507d63bd79d5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:50.747761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44032338-a051-4856-8336-507d63bd79d5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:50.786763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:50.788090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:50.788517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:50.788956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:50.802308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:50.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:50.808589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:50.809597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:50.813140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 195/385] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:50.818292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:50.901337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:50.952231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:50.985496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:50.993976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec5966-c699-474a-9f9f-99e76ee09afe None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.021360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8593-2ab2-4a16-a54e-2492a6ee1789 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.051137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.052152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.052724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.053296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.054168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:51.054680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:51.056298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:51.060938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.060938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.060938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.060938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.072682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:51.075329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 191/386] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:51.079109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:51.080159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:51.081423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 196/387] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:51.082896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:51.086454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:51.188110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:51.194820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68bf01-9cef-44a1-914c-10062912a62d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.231248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.231898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.232187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.232465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.236070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:51.236070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:51.236070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:51.244653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 197/388] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:51.252879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:51.277844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27ca1a-994c-4961-b935-b7941d095539 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.362447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea8626-2090-4d1e-8cbd-d1ab6cfc161e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.384544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.390371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.390371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.390640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.402116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.402346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:51.402791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.403074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.403353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.403680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:51.404625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:51.406095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 192/389] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:51 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:51.406331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:51.408384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 198/390] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:51 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:51.412035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:51.413908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:51.452345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:51.488918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:51.531364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e68b44-6fc1-4304-b412-3267d7f8d0cb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.572703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.573855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.574157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.574458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.577113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:51.578629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:51.581657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 199/391] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:51 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:51.587655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:51.697851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8fae2-5644-4398-803c-101e285aa310 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.698503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:51.708658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5adb1c-4919-4d86-80db-690cf4f913d3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:51.753828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d1b2a-dbfd-471d-aef6-21792560bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.754511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1b2a-dbfd-471d-aef6-21792560bf81 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.755627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1b2a-dbfd-471d-aef6-21792560bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.755941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1b2a-dbfd-471d-aef6-21792560bf81 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392/groups/3472354601c1437eab2c2ef020cd1e79/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.760572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d1b2a-dbfd-471d-aef6-21792560bf81 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f87007362fc43378dacfe7a12000392, group_id=3472354601c1437eab2c2ef020cd1e79, role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:51.762020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d1b2a-dbfd-471d-aef6-21792560bf81 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:51.773484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:51.774339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:51.774630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:51.774999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:51.776801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:51.778460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:51.780634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 193/392] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:51 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:51.788125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:51.795938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 200/393] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Oct 6 10:50:51 2022] PUT /identity/v3/projects/7f87007362fc43378dacfe7a12000392/groups/3472354601c1437eab2c2ef020cd1e79/roles/2372f2ee163c430ca4b943c833677192 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:51.962895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:51.976380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f446c-3a2d-4151-bdc3-9a61fef96544 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:52.027246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8822264-04a7-4b41-9890-457aa4b296d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:52.028038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822264-04a7-4b41-9890-457aa4b296d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:52.028232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822264-04a7-4b41-9890-457aa4b296d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:52.028507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822264-04a7-4b41-9890-457aa4b296d5 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392/groups/3472354601c1437eab2c2ef020cd1e79/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:52.030404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822264-04a7-4b41-9890-457aa4b296d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f87007362fc43378dacfe7a12000392, group_id=3472354601c1437eab2c2ef020cd1e79, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:52.032128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822264-04a7-4b41-9890-457aa4b296d5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:52.057689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 194/394] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Oct 6 10:50:51 2022] PUT /identity/v3/projects/7f87007362fc43378dacfe7a12000392/groups/3472354601c1437eab2c2ef020cd1e79/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:52.365686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec854d1-3a4e-4cb3-b426-fcbb97934120 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:52.366374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec854d1-3a4e-4cb3-b426-fcbb97934120 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:52.366883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec854d1-3a4e-4cb3-b426-fcbb97934120 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:52.368811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 201/395] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:52.373910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a965-4b2a-4d40-b7cc-664f8f04e543 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:52.374192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a965-4b2a-4d40-b7cc-664f8f04e543 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:52.374470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a965-4b2a-4d40-b7cc-664f8f04e543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:52.397266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d506a965-4b2a-4d40-b7cc-664f8f04e543 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:52.398468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d506a965-4b2a-4d40-b7cc-664f8f04e543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:52.477761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 195/396] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:52.974294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec854d1-3a4e-4cb3-b426-fcbb97934120 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:53.109512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec854d1-3a4e-4cb3-b426-fcbb97934120 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:53.140250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101f35d-a6ee-4d19-99e9-3c796151610b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.141217 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101f35d-a6ee-4d19-99e9-3c796151610b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.141743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101f35d-a6ee-4d19-99e9-3c796151610b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.143280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 196/397] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:53.149039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea5f23-d368-4830-95ce-11963c6bdcaa None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.149585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea5f23-d368-4830-95ce-11963c6bdcaa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.150069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea5f23-d368-4830-95ce-11963c6bdcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.161169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:53.162113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.162604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.163066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.180220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2eea5f23-d368-4830-95ce-11963c6bdcaa None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:53.182904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2eea5f23-d368-4830-95ce-11963c6bdcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:53.190076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:53.198081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:53.198983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:53.199774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:53.208035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 202/398] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:53.213528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:53.242912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:53.287388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 197/399] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:53.387708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647018d-42a8-44c4-8fda-ac5cde290caa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:53.445365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1e0e7-f971-449a-938c-aa22644c1197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:53.445997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1e0e7-f971-449a-938c-aa22644c1197 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.446283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1e0e7-f971-449a-938c-aa22644c1197 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.446557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1e0e7-f971-449a-938c-aa22644c1197 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.447236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1e0e7-f971-449a-938c-aa22644c1197 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:53.452217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1e0e7-f971-449a-938c-aa22644c1197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:53.453639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1e0e7-f971-449a-938c-aa22644c1197 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:53.462994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 203/400] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:53.468824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea5f23-d368-4830-95ce-11963c6bdcaa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:53.490741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:53.550612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423a8f8-b58b-40f9-abf3-faabcf55b132 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.551132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423a8f8-b58b-40f9-abf3-faabcf55b132 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.551616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423a8f8-b58b-40f9-abf3-faabcf55b132 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.553492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 204/401] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:53.558334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5427-1f58-40da-baca-634ba8506516 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.559583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5427-1f58-40da-baca-634ba8506516 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.560108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5427-1f58-40da-baca-634ba8506516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.597224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e18d5427-1f58-40da-baca-634ba8506516 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:53.598969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e18d5427-1f58-40da-baca-634ba8506516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:53.599755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea5f23-d368-4830-95ce-11963c6bdcaa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:53.640463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:53.641439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.642563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.643030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.644614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:53.646618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:53.649148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 198/402] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:53 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:53.654658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:53.708209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 205/403] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:53.709431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d5427-1f58-40da-baca-634ba8506516 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:53.718899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:53.753260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:53.791729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58652a7d-3494-462f-a8c2-7165d2b7e42a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:53.854814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:53.855363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.855637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.855936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.857820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:53.859260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:53.861919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 199/404] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:53 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:53.871869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:53.884833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d5427-1f58-40da-baca-634ba8506516 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:53.950603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:53.951289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:53.951570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:53.951850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:53.972782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:53.982168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:53.983577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:53.984411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 206/405] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:53.992550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.054140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d369e4f-fef0-4418-be6c-ecb4752d4fb0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.091809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.093057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:54.094008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.095667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.096153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25/groups/3472354601c1437eab2c2ef020cd1e79/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.099137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=994f81afb7594f73a78a9d847a544f25, group_id=3472354601c1437eab2c2ef020cd1e79, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:54.101967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:54.121553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 200/406] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Oct 6 10:50:53 2022] PUT /identity/v3/projects/994f81afb7594f73a78a9d847a544f25/groups/3472354601c1437eab2c2ef020cd1e79/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:54.128944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.149858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:54.154010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeec8b7-9421-4230-a6f5-aca4df743418 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.233718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.236264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.236264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.236264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.236264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:54.239477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:54.241085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:54.255968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 207/407] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:54.268168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.342041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fef5af-eae1-40f9-ac61-6b0e608bb6f9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.401362 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efafe32-354d-4838-8590-ff89d83677ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.402009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efafe32-354d-4838-8590-ff89d83677ff None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.402292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efafe32-354d-4838-8590-ff89d83677ff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.402564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efafe32-354d-4838-8590-ff89d83677ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.416555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efafe32-354d-4838-8590-ff89d83677ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6efafe32-354d-4838-8590-ff89d83677ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:54.424106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:54.425011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:54.425834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:50:54.428918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 201/408] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:54.430912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b66491-75c4-4eb2-80b4-78753bfbf8be None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.431427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b66491-75c4-4eb2-80b4-78753bfbf8be None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.432340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b66491-75c4-4eb2-80b4-78753bfbf8be None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.434974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 202/409] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:54.440605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b66491-75c4-4eb2-80b4-78753bfbf8be None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.457953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaae26-5c6b-41dd-a371-6448034897ba None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.493854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:54.505619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b0c8b-6811-4d01-bd85-da3d8ff499d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.507634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b0c8b-6811-4d01-bd85-da3d8ff499d7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.507966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b0c8b-6811-4d01-bd85-da3d8ff499d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.508249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b0c8b-6811-4d01-bd85-da3d8ff499d7 None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.509683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b0c8b-6811-4d01-bd85-da3d8ff499d7 None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:54.511469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b0c8b-6811-4d01-bd85-da3d8ff499d7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:54.513950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 208/410] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:54 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:54.519710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30948b-ec23-4c97-ad44-0058caf04407 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.519710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30948b-ec23-4c97-ad44-0058caf04407 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.519710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30948b-ec23-4c97-ad44-0058caf04407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.538674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:54.541838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-af30948b-ec23-4c97-ad44-0058caf04407 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:54.543261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af30948b-ec23-4c97-ad44-0058caf04407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:54.576943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b66491-75c4-4eb2-80b4-78753bfbf8be None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.615098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.615753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.616057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.616332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.616994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:54.617537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:54.618753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:54.622568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 209/411] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:54.623685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30948b-ec23-4c97-ad44-0058caf04407 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.633343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 203/412] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:54.638669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.744997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30948b-ec23-4c97-ad44-0058caf04407 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.754553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4c57d-dd6b-4b15-983a-6993a7cc3e32 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.773410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:54.788204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:54.791977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.792836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.793122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.793307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.793412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.794045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.794334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.794549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:54.794646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] PATH_INFO: `/v3/groups/d1fdb6ae834e4efdae606e0e04d6c74e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.795789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] RBAC: Authorizing `identity:get_group(group_id=d1fdb6ae834e4efdae606e0e04d6c74e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:54.796133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:54.798106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 210/413] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:54 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:54.803447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:54.803816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.805231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 204/414] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:54 2022] GET /identity/v3/groups/d1fdb6ae834e4efdae606e0e04d6c74e => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:54.810533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:54.914105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63988a-71a3-44ef-b846-39ede8cb43b6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.965652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f27c3-97f0-4b47-b792-6ad5541727ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:54.966373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f27c3-97f0-4b47-b792-6ad5541727ee None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.966662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f27c3-97f0-4b47-b792-6ad5541727ee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.966973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f27c3-97f0-4b47-b792-6ad5541727ee None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25/groups/3472354601c1437eab2c2ef020cd1e79/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:54.969204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f27c3-97f0-4b47-b792-6ad5541727ee None admin] RBAC: Authorizing `identity:create_grant(project_id=994f81afb7594f73a78a9d847a544f25, group_id=3472354601c1437eab2c2ef020cd1e79, role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:54.971272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f27c3-97f0-4b47-b792-6ad5541727ee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:54.980921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ab896-6c2b-4825-8b50-4d7792cd11db None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:54.984919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 211/415] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Oct 6 10:50:54 2022] PUT /identity/v3/projects/994f81afb7594f73a78a9d847a544f25/groups/3472354601c1437eab2c2ef020cd1e79/roles/2372f2ee163c430ca4b943c833677192 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 10:50:54.986320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013028a2-3ca7-4506-bcf8-c529d989df51 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:54.986702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013028a2-3ca7-4506-bcf8-c529d989df51 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:54.986996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013028a2-3ca7-4506-bcf8-c529d989df51 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.000798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 212/416] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:55.001526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013028a2-3ca7-4506-bcf8-c529d989df51 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:55.020882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252aecc-b99b-4025-ae63-4971f0bdb51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:55.021280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252aecc-b99b-4025-ae63-4971f0bdb51b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.021473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252aecc-b99b-4025-ae63-4971f0bdb51b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.022526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252aecc-b99b-4025-ae63-4971f0bdb51b None admin] PATH_INFO: `/v3/projects/46752b92e78642e4b79f5ef50f2e51da` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.023564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252aecc-b99b-4025-ae63-4971f0bdb51b None admin] RBAC: Authorizing `identity:get_project(project_id=46752b92e78642e4b79f5ef50f2e51da)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:55.025275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252aecc-b99b-4025-ae63-4971f0bdb51b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:55.028152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 205/417] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:54 2022] GET /identity/v3/projects/46752b92e78642e4b79f5ef50f2e51da => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:55.029958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5371-e3dd-4c38-ac0f-2cdb6e282a14 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.030241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5371-e3dd-4c38-ac0f-2cdb6e282a14 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.030512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5371-e3dd-4c38-ac0f-2cdb6e282a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.055169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fe4c5371-e3dd-4c38-ac0f-2cdb6e282a14 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:55.056448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fe4c5371-e3dd-4c38-ac0f-2cdb6e282a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:55.076426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:55.126460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013028a2-3ca7-4506-bcf8-c529d989df51 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:55.146402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:55.159423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 206/418] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:55.160472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c5371-e3dd-4c38-ac0f-2cdb6e282a14 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:55.166408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:55.167045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.167320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.167589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.181256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:55.186211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:55.187119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:55.188383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 213/419] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:55.194367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:55.307150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c5371-e3dd-4c38-ac0f-2cdb6e282a14 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:55.357221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aea62-2cb4-41a2-afe8-a207d4b4a4c4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:55.373458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:55.375007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:55.376446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.377071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.378481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] PATH_INFO: `/v3/projects/46752b92e78642e4b79f5ef50f2e51da/groups/d1fdb6ae834e4efdae606e0e04d6c74e/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.380886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] RBAC: Authorizing `identity:create_grant(project_id=46752b92e78642e4b79f5ef50f2e51da, group_id=d1fdb6ae834e4efdae606e0e04d6c74e, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:55.382889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:55.406387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:55.407138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.407644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.409765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.410486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:55.410802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:55.412428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:55.414216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 207/420] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Oct 6 10:50:55 2022] PUT /identity/v3/projects/46752b92e78642e4b79f5ef50f2e51da/groups/d1fdb6ae834e4efdae606e0e04d6c74e/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:50:55.428615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 214/421] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:55.436610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:55.512210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:55.546283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:55.634283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bb356-ea36-4e84-9fae-86a0b1f30195 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:55.679614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c4dd3-8f23-4bdd-abce-29e1bdff2161 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:55.685806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:55.686727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.687353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.687628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.718922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:55.720068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.720906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.721394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:55.733518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:55.740523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:55.741509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:55.742292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 215/422] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:55.749033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:55.808195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:55.809676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:55.816179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:55.917289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:55.931974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf797e5-a68e-4e00-9919-398f4ce6d369 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:55.997318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:55.997987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:55.998305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:55.998582 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.000328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:56.000608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 208/423] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:56.000973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:56.002694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:56.014595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:56.015681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 216/424] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:56.023790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:56.194897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:56.199879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586f9d-9d07-4a5d-855a-721ce54eacd3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:56.209961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ed56-9720-45e3-9093-d1dbd6f42f38 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:56.263898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:56.264516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:56.264802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:56.265074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.271330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:56.272059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:56.272341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:56.272611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.273864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:56.275300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:56.276948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 209/425] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:56 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:56.285292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:56.332897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:56.391864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:56.394772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:56.468871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5be452-00e4-490a-970f-3be1f10f8637 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:56.535173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:56.535823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:56.536122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:56.536393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.537516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:56.538965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:56.540934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 210/426] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:56 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:56.553055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:56.555405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:56.570819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 217/427] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:50:56.576520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:56.732720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:56.745805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84293f10-768d-41d2-9b44-a8b945f97cdb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:56.754732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824106ba-40ae-4820-98e7-25b71741dabd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:56.781938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ced45-e043-4fe4-aa33-705648ae1150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:56.783091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ced45-e043-4fe4-aa33-705648ae1150 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:56.783525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ced45-e043-4fe4-aa33-705648ae1150 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:56.783801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ced45-e043-4fe4-aa33-705648ae1150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.784751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ced45-e043-4fe4-aa33-705648ae1150 None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=7f87007362fc43378dacfe7a12000392) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:56.785017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ced45-e043-4fe4-aa33-705648ae1150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:56.786572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ced45-e043-4fe4-aa33-705648ae1150 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:56.793341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 211/428] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:56 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=2372f2ee163c430ca4b943c833677192 => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:56.794690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398f10f-54cc-434d-8cb3-199a173f9dde None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:56.794972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398f10f-54cc-434d-8cb3-199a173f9dde None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:56.799884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398f10f-54cc-434d-8cb3-199a173f9dde None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.799884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 212/429] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:56.804014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:56.805738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840e13-4567-4722-bc2a-fcf4cd9cc742 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:56.806161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840e13-4567-4722-bc2a-fcf4cd9cc742 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:56.806559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840e13-4567-4722-bc2a-fcf4cd9cc742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.810366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:56.810890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:56.811309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] PATH_INFO: `/v3/projects/7f87007362fc43378dacfe7a12000392` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:56.812869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] RBAC: Authorizing `identity:get_project(project_id=7f87007362fc43378dacfe7a12000392)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:56.815073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:56.817438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 218/430] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:56 2022] GET /identity/v3/projects/7f87007362fc43378dacfe7a12000392 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:56.823215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:56.850788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-da840e13-4567-4722-bc2a-fcf4cd9cc742 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:56.854792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-da840e13-4567-4722-bc2a-fcf4cd9cc742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:56.989984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:56.992051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 213/431] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:50:57.035672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75205b-e263-45be-97f3-58ece239dbb9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:57.071362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:57.072032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.072314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.072589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.073705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:57.075149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:57.077103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 219/432] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:56 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:57.085172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da840e13-4567-4722-bc2a-fcf4cd9cc742 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:57.179626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:57.196025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da840e13-4567-4722-bc2a-fcf4cd9cc742 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:57.231133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:57.231742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.232027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.232353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.233072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=7f87007362fc43378dacfe7a12000392) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:57.233371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:57.234895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:57.241906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 214/433] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:57 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=7f87007362fc43378dacfe7a12000392&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:57.334234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:57.458932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988709a-57f7-4311-8dfe-cfeb9215b098 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:57.494822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f02f8-b747-4a73-adae-4d1a4ac0bef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:57.495477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f02f8-b747-4a73-adae-4d1a4ac0bef6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.495752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f02f8-b747-4a73-adae-4d1a4ac0bef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.496091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f02f8-b747-4a73-adae-4d1a4ac0bef6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.498337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:57.509889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f02f8-b747-4a73-adae-4d1a4ac0bef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b29f02f8-b747-4a73-adae-4d1a4ac0bef6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:57.514956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:57.515965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:57.517796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 220/434] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:57.523193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:57.625544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404e9e-72ba-4e8e-a8f5-96712120e547 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.626373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404e9e-72ba-4e8e-a8f5-96712120e547 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.626723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404e9e-72ba-4e8e-a8f5-96712120e547 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.628520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 221/435] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:50:57.634127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def23317-21ca-4e1e-ae43-f05c3591a331 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.634500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def23317-21ca-4e1e-ae43-f05c3591a331 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.634657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def23317-21ca-4e1e-ae43-f05c3591a331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.636160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc74a3f-14c7-448c-b76b-af64dde78e11 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:57.659289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-def23317-21ca-4e1e-ae43-f05c3591a331 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:57.660882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-def23317-21ca-4e1e-ae43-f05c3591a331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:57.686439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:57.686439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.686439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.686439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.686829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:57.686829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:57.688179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:57.699681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 215/436] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:50:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:57.705916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:57.730364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:57.763920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 222/437] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:57.840692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4f8bb-74db-4177-ab7c-beeaaa4a9560 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:57.876979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:57.877588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.877869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.878141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.888505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f6d9-40c0-487c-926d-9cef78cc98ac None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.888805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f6d9-40c0-487c-926d-9cef78cc98ac None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.889064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f6d9-40c0-487c-926d-9cef78cc98ac None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.892090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 223/438] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:50:57.896333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e223f-8ec4-44b2-a69c-378052b3c00e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:57.896618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e223f-8ec4-44b2-a69c-378052b3c00e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:57.896898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e223f-8ec4-44b2-a69c-378052b3c00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:57.922479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-af9e223f-8ec4-44b2-a69c-378052b3c00e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:50:57.925311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af9e223f-8ec4-44b2-a69c-378052b3c00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:50:57.962386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:57.964261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:57.979128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.005268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 224/439] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:50:58.036950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e223f-8ec4-44b2-a69c-378052b3c00e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.072901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 216/440] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:50:58.078195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.097460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.146915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e223f-8ec4-44b2-a69c-378052b3c00e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.180415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.181152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.181436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.181725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.184794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f0826-6804-4ea5-b6d5-5fbb6b942471 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.192777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:58.197846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:58.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:58.199825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 225/441] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:50:58.202032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.205506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.226851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.227809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.228352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.228817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.230151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:58.231771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:58.233686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 217/442] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:58.239172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.317263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e53ec1-abf1-4ca9-a1bc-22103c206bad None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.331242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.340865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a215cb-46e4-4ce1-8953-f158043f0faf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.351617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.352397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.352679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.352954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.353613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:58.353916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:58.355287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:58.363873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 226/443] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:58.368596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.377228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.377856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.378139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.378412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.379538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:58.381002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:58.387241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 218/444] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:58.387241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.440833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.467754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e7274-d137-4662-b340-7a90a0594a7f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.492264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151db2c5-2ecc-4377-ba49-55bbfed45c16 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.502372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.503000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.503283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.503560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.513828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:50:58.520775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:50:58.521846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:50:58.522675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 227/445] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:50:58.524862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.527697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.528343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.528623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.528892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.560619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.595928 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:58.597371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:58.625047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5f31-1dff-4c51-aa97-c42655fc2e43 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.643389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.659948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.660552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.660835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.661108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.661839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=1a3c6cbe3fb44e9a812652129507717e, scope.project.id=994f81afb7594f73a78a9d847a544f25) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:58.662150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:58.663598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:58.669939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 228/446] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=1a3c6cbe3fb44e9a812652129507717e => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:58.670719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.701771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 219/447] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:50:58.703048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.796559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.802930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2f0c5-aa4b-4894-8e2c-cf3c83dd6a7b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.811201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d08a-b4ec-495f-9b19-f8698008ddc1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.842734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.843784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.843784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.843982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.845164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:58.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:58.846887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:58.848373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:58.849011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:58.850002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:58.850002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] PATH_INFO: `/v3/projects/994f81afb7594f73a78a9d847a544f25` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:58.851810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] RBAC: Authorizing `identity:get_project(project_id=994f81afb7594f73a78a9d847a544f25)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:58.852129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:58.854033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 220/448] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/projects/994f81afb7594f73a78a9d847a544f25 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:58.860156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.861323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 229/449] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:58.866687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:58.883691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:58.988008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178eef3-0d42-443e-8a99-ce9238a04b25 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:58.993804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680a0c6-5de9-4d98-afab-6ff06df3079c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:59.029668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:59.030298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:59.030824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:59.031095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] PATH_INFO: `/v3/groups/3472354601c1437eab2c2ef020cd1e79` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:59.032813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] RBAC: Authorizing `identity:get_group(group_id=3472354601c1437eab2c2ef020cd1e79)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:59.034247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:59.035828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 221/450] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/groups/3472354601c1437eab2c2ef020cd1e79 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:59.036031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:59.036652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:59.036985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:59.037519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:59.042329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:59.070817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:59.109255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:59.111195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:59.130282 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:59.145862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b1e1-f365-43c6-b59a-29b92b8cdf44 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:59.180303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:59.180898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:59.181171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:59.181455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:59.182148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] RBAC: Adding query filter params (group.id=3472354601c1437eab2c2ef020cd1e79, role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=994f81afb7594f73a78a9d847a544f25) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:59.182452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:59.184323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:59.190638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 222/451] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:59 2022] GET /identity/v3/role_assignments?group.id=3472354601c1437eab2c2ef020cd1e79&scope.project.id=994f81afb7594f73a78a9d847a544f25&role.id=2372f2ee163c430ca4b943c833677192 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:59.210332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 230/452] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:50:59.216238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:59.322327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43126b4f-8127-48f5-a11d-1d7ce19ef8fd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:59.358707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:59.359395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:59.359622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:59.359923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] PATH_INFO: `/v3/projects/46752b92e78642e4b79f5ef50f2e51da` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:59.361637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] RBAC: Authorizing `identity:get_project(project_id=46752b92e78642e4b79f5ef50f2e51da)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:59.363068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:59.364719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 223/453] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:50:59 2022] GET /identity/v3/projects/46752b92e78642e4b79f5ef50f2e51da => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:50:59.370551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:59.433199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:59.471467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca3fa-582e-45c9-87fe-4c7fef51bbaf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:59.506460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50d66a-b5ce-4bc6-a3cc-6d4471c5ab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:59.507091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50d66a-b5ce-4bc6-a3cc-6d4471c5ab8d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:59.507372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50d66a-b5ce-4bc6-a3cc-6d4471c5ab8d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:59.507643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50d66a-b5ce-4bc6-a3cc-6d4471c5ab8d None admin] PATH_INFO: `/v3/groups/d1fdb6ae834e4efdae606e0e04d6c74e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:59.508806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50d66a-b5ce-4bc6-a3cc-6d4471c5ab8d None admin] RBAC: Authorizing `identity:get_group(group_id=d1fdb6ae834e4efdae606e0e04d6c74e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:59.510250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50d66a-b5ce-4bc6-a3cc-6d4471c5ab8d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:59.511824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 231/454] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Oct 6 10:50:59 2022] GET /identity/v3/groups/d1fdb6ae834e4efdae606e0e04d6c74e => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:50:59.518054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:50:59.537278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:50:59.618764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba465b7-a70d-45b5-b55a-df7e4b1a24c6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:50:59.652434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fcd38-033b-4657-b63a-617fe3768c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:50:59.653443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fcd38-033b-4657-b63a-617fe3768c6c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:50:59.653942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fcd38-033b-4657-b63a-617fe3768c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:50:59.654429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fcd38-033b-4657-b63a-617fe3768c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:50:59.656081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fcd38-033b-4657-b63a-617fe3768c6c None admin] RBAC: Adding query filter params (group.id=d1fdb6ae834e4efdae606e0e04d6c74e, role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=46752b92e78642e4b79f5ef50f2e51da) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:50:59.656640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fcd38-033b-4657-b63a-617fe3768c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:50:59.659469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fcd38-033b-4657-b63a-617fe3768c6c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:50:59.665349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 224/455] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Oct 6 10:50:59 2022] GET /identity/v3/role_assignments?group.id=d1fdb6ae834e4efdae606e0e04d6c74e&scope.project.id=46752b92e78642e4b79f5ef50f2e51da&role.id=4c41d809cc274ff5871dc8092f05715a => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:02.409685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4f76d-2811-452d-ac33-cbb7786ee286 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:02.410312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4f76d-2811-452d-ac33-cbb7786ee286 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:02.410312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4f76d-2811-452d-ac33-cbb7786ee286 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:02.411823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 232/456] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:02.417294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09215a2e-ef0f-42fe-9931-afb16bc3e76a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:02.417815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09215a2e-ef0f-42fe-9931-afb16bc3e76a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:02.417815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09215a2e-ef0f-42fe-9931-afb16bc3e76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:02.443030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-09215a2e-ef0f-42fe-9931-afb16bc3e76a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:02.444252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-09215a2e-ef0f-42fe-9931-afb16bc3e76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:02.543077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 225/457] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:02.634729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad820f8a-7fcd-41c2-8e1e-81e384a28c27 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:02.635271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad820f8a-7fcd-41c2-8e1e-81e384a28c27 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:02.635705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad820f8a-7fcd-41c2-8e1e-81e384a28c27 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:02.637189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 233/458] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:02.643200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20952a8-1ec7-4574-9f4c-013b485f2af7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:02.643711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20952a8-1ec7-4574-9f4c-013b485f2af7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:02.644200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20952a8-1ec7-4574-9f4c-013b485f2af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:02.646772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f042d-0cd1-4105-b3e8-b5468f2e8d66 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:02.647270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f042d-0cd1-4105-b3e8-b5468f2e8d66 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:02.647754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f042d-0cd1-4105-b3e8-b5468f2e8d66 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:02.649222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 234/459] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:02.654413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a692dd-e307-4c09-b75f-cf4e15fb5588 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:02.654696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a692dd-e307-4c09-b75f-cf4e15fb5588 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:02.654974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a692dd-e307-4c09-b75f-cf4e15fb5588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:02.677597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-13a692dd-e307-4c09-b75f-cf4e15fb5588 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:02.679112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-13a692dd-e307-4c09-b75f-cf4e15fb5588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:02.680826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a20952a8-1ec7-4574-9f4c-013b485f2af7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:02.681973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a20952a8-1ec7-4574-9f4c-013b485f2af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:02.684956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:02.789344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 235/460] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:51:02.825069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 226/461] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:02.899071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a692dd-e307-4c09-b75f-cf4e15fb5588 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:02.970731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:03.029381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a692dd-e307-4c09-b75f-cf4e15fb5588 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.075563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.076603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.077087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.077560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.098168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.100027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:03.120986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:03.121981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:03.123343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:03.123343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:03.123343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:03.124218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20952a8-1ec7-4574-9f4c-013b485f2af7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.125206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 236/462] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:03.135080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.270908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa39c8e-1b42-4d57-9ff2-f76e9d60cbef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.272823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20952a8-1ec7-4574-9f4c-013b485f2af7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.288618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:03.316144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c4d2d-1443-4861-8e85-0d96d72fd370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.317164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c4d2d-1443-4861-8e85-0d96d72fd370 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.317639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c4d2d-1443-4861-8e85-0d96d72fd370 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.317742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.318272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c4d2d-1443-4861-8e85-0d96d72fd370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.318614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.318904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.319186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.319857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:03.320181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.321560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:03.333817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:03.334027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 237/463] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:03.335195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.337916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c4d2d-1443-4861-8e85-0d96d72fd370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.341325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c4d2d-1443-4861-8e85-0d96d72fd370 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-a57c4d2d-1443-4861-8e85-0d96d72fd370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:03.363416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:03.364401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 227/464] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757e6e9-5b1e-4a1d-9536-9cab38a42ab9 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757e6e9-5b1e-4a1d-9536-9cab38a42ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757e6e9-5b1e-4a1d-9536-9cab38a42ab9 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 228/465] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:03.372028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757e6e9-5b1e-4a1d-9536-9cab38a42ab9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.463417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab40e3-e4d1-4197-ae91-fb469696ed6e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.507912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757e6e9-5b1e-4a1d-9536-9cab38a42ab9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.510110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.510828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.511154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.511899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.524993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.526851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:03.546802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:03.548182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:03.549094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:03.549094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:03.549094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:03.549266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6869f-997c-47cb-b3c7-e00a3b922e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.549266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6869f-997c-47cb-b3c7-e00a3b922e52 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.549266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6869f-997c-47cb-b3c7-e00a3b922e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.549466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 238/466] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:03.549534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6869f-997c-47cb-b3c7-e00a3b922e52 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.549534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6869f-997c-47cb-b3c7-e00a3b922e52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.549649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.550788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6869f-997c-47cb-b3c7-e00a3b922e52 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:03.576116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 229/467] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:51:03 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:03.577789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6ac5e-814e-492f-8b1e-90bdb3dae861 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.578085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6ac5e-814e-492f-8b1e-90bdb3dae861 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.578364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6ac5e-814e-492f-8b1e-90bdb3dae861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.587275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:03.606357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3fa6ac5e-814e-492f-8b1e-90bdb3dae861 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:03.607698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3fa6ac5e-814e-492f-8b1e-90bdb3dae861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:03.675947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:03.712910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573d55f-4d44-40ca-814c-efdf84fbe4c9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.728374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 230/468] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:03.730021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6ac5e-814e-492f-8b1e-90bdb3dae861 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.765965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.767150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.767428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.767699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.768403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:03.768708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.770904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:03.804616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 239/469] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:03.811041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.896538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6ac5e-814e-492f-8b1e-90bdb3dae861 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.928232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:03.946220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c680bf9-3c8f-4a26-a24a-d27a85b3cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.947422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c680bf9-3c8f-4a26-a24a-d27a85b3cb71 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.947713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c680bf9-3c8f-4a26-a24a-d27a85b3cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.948024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c680bf9-3c8f-4a26-a24a-d27a85b3cb71 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.948680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c680bf9-3c8f-4a26-a24a-d27a85b3cb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:03.948984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c680bf9-3c8f-4a26-a24a-d27a85b3cb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.949210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a94e0-feff-4410-8a13-7557513abee6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:03.950378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c680bf9-3c8f-4a26-a24a-d27a85b3cb71 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:03.963780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 231/470] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:03.965189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c83206-ec7b-4648-9db5-57a45a81f3d4 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.965580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c83206-ec7b-4648-9db5-57a45a81f3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.965955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c83206-ec7b-4648-9db5-57a45a81f3d4 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.967259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 232/471] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:03.972468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c83206-ec7b-4648-9db5-57a45a81f3d4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:03.988033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975cf10-c76e-4df8-ae60-a805395aadf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:03.989218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cf10-c76e-4df8-ae60-a805395aadf0 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:03.989888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cf10-c76e-4df8-ae60-a805395aadf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:03.990313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cf10-c76e-4df8-ae60-a805395aadf0 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:03.991187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975cf10-c76e-4df8-ae60-a805395aadf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:03.992970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975cf10-c76e-4df8-ae60-a805395aadf0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:04.017364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:04.020609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 240/472] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:51:03 2022] POST /identity/v3/users => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:51:04.023115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdfed-c5ab-44e6-ae80-fd86f70a4e70 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:04.023595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdfed-c5ab-44e6-ae80-fd86f70a4e70 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:04.024030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdfed-c5ab-44e6-ae80-fd86f70a4e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:04.048498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-64bbdfed-c5ab-44e6-ae80-fd86f70a4e70 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:04.050216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-64bbdfed-c5ab-44e6-ae80-fd86f70a4e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:04.091907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c83206-ec7b-4648-9db5-57a45a81f3d4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:04.137382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:04.138214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:04.138598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:04.138968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:04.139785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:04.141576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:04.148551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 241/473] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:04.149645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbdfed-c5ab-44e6-ae80-fd86f70a4e70 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:04.186783 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 233/474] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:51:03 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:04.257045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:04.275981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbdfed-c5ab-44e6-ae80-fd86f70a4e70 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:04.316941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:04.317838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:04.318289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:04.318699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:04.330803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:04.332631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:04.351576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:04.352755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:04.353813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:04.353813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:04.353813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:04.355317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 242/475] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:04.361878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:04.465533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:04.482887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:04.492717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f66484-5d34-4aa3-8897-f0168670f3a9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:04.540778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:04.541549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:04.541840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:04.542120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:04.542770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:04.543588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:04.545333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:04.564433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 234/476] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:04.570134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:04.612615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36d473-fc30-44ad-b879-73ecfd6efe05 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:04.636744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:04.649842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:04.650868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:04.651393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:04.651898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:04.652790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:04.654354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:04.670597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 10:51:04.670597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 06 10:51:04.670597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [parameters: {'id': 'ea03a09f4c154622905fd314bf82bd07', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 06 10:51:04.670597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Oct 06 10:51:04.704708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return ref.to_dict() Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 10:51:04.705745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 10:51:04.706668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 10:51:04.707573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 10:51:04.708526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 10:51:04.709382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'id': 'ea03a09f4c154622905fd314bf82bd07', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:04.710191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 06 10:51:04.711076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:04.717567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 243/477] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Oct 6 10:51:04 2022] POST /identity/v3/roles => generated 144 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:04.717710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211859f-8f32-48fa-8229-5911ab7f241e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:04.720172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:04.784690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:04.785313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:04.785600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:04.786817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:04.787442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:04.789599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:04.831563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 235/478] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:51:04 2022] POST /identity/v3/users => generated 249 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:04.914216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384981e-e479-4420-86d3-8470402c3ee8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:04.949507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383bc4c-f933-4418-9ba6-9a23375ae142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:04.950146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383bc4c-f933-4418-9ba6-9a23375ae142 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:04.950423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383bc4c-f933-4418-9ba6-9a23375ae142 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:04.950701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383bc4c-f933-4418-9ba6-9a23375ae142 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:04.962983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383bc4c-f933-4418-9ba6-9a23375ae142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1383bc4c-f933-4418-9ba6-9a23375ae142 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:04.968276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:04.969710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 10:51:04.970597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:04.971232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 244/479] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:04.978020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:05.000300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:05.115662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2ee47-e989-414a-ae1a-a70f1c25828f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:05.160197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811290e9-bc81-41d3-b6a0-db56f91816da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:05.160815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811290e9-bc81-41d3-b6a0-db56f91816da None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:05.161098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811290e9-bc81-41d3-b6a0-db56f91816da None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:05.161376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811290e9-bc81-41d3-b6a0-db56f91816da None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:05.162175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811290e9-bc81-41d3-b6a0-db56f91816da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:05.162488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811290e9-bc81-41d3-b6a0-db56f91816da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:05.163884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811290e9-bc81-41d3-b6a0-db56f91816da None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:05.173516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 236/480] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:06.316259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da99daf-3eed-4724-a323-f2964fe68ba9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:06.317341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da99daf-3eed-4724-a323-f2964fe68ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:06.317900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da99daf-3eed-4724-a323-f2964fe68ba9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:06.319688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 245/481] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:06.325242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154167-ea7b-4276-a625-bf15d1631633 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:06.325648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154167-ea7b-4276-a625-bf15d1631633 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:06.325815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154167-ea7b-4276-a625-bf15d1631633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:06.347561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d0154167-ea7b-4276-a625-bf15d1631633 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:06.348756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d0154167-ea7b-4276-a625-bf15d1631633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:06.421159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 237/482] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:06.779147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf742-72ae-4038-b315-30600d645db3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:06.779431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf742-72ae-4038-b315-30600d645db3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:06.779707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf742-72ae-4038-b315-30600d645db3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:06.781616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 246/483] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:06.787146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c903f-13f1-4abe-adb6-4679225a2768 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:06.787431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c903f-13f1-4abe-adb6-4679225a2768 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:06.787713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c903f-13f1-4abe-adb6-4679225a2768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:06.794262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:06.810732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4c3c903f-13f1-4abe-adb6-4679225a2768 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:06.811934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4c3c903f-13f1-4abe-adb6-4679225a2768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:06.868886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbf742-72ae-4038-b315-30600d645db3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:06.896117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 238/484] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:51:06.936441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:06.999232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbf742-72ae-4038-b315-30600d645db3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.035246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.035799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.036097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.036368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.047118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:07.052406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:07.053521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:07.054793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 247/485] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:07.055454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dca85-806d-4ee8-8ab0-6d52bc8654c9 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.055454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dca85-806d-4ee8-8ab0-6d52bc8654c9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.055454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dca85-806d-4ee8-8ab0-6d52bc8654c9 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.056893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 239/486] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:07.060925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.065140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434c602-3911-4925-828b-9b7c903bf3a8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.065673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434c602-3911-4925-828b-9b7c903bf3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.066138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434c602-3911-4925-828b-9b7c903bf3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.104818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6434c602-3911-4925-828b-9b7c903bf3a8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:07.106340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6434c602-3911-4925-828b-9b7c903bf3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:07.164269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c39eb2-61e5-4dc8-989e-33a4ae2698a5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.171298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:07.203882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.203882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.203882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.203882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.204256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:07.204256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:07.204256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:07.211852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 248/487] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:07.217190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.253382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 240/488] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:51:07.254379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434c602-3911-4925-828b-9b7c903bf3a8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.324448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:07.366718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434c602-3911-4925-828b-9b7c903bf3a8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.401852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037c395-f9b7-4811-88cd-dca650bec80d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.431529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.432182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.432460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.432728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.461043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:07.464975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.465658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.465937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.466205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:07.471115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:07.472119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:07.472931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:07.477684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 241/489] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:07.478776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.490528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:07.492002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:07.522599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:07.524884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:07.526877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:07.526877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:07.526877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:07.526877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 249/490] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:07.529501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.591092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:07.619190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8353b7-4bef-4048-85c6-e8af1f3a114a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.643935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:07.657531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.658133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.658412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.658685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.659345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:07.659643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:07.661129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:07.670109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 242/491] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:07.670911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.714704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647e8db-ea82-4f68-965b-875e2f763c87 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.757935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.758549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.758829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.759103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.759750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:07.760089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:07.761469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:07.775590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 250/492] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:07.776684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.789983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e205e-1616-45c3-b83b-a79aa193bb94 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.825396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.826027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.826304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.826570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.837623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:07.843189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:07.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:07.845466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 243/493] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:07.846164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.873248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:07.892101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:07.896122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064fb9d-36ee-40a1-900d-f2fe0eb6c0eb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.933920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ddd0e-aa66-4667-95a3-2d04e7780840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.934554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ddd0e-aa66-4667-95a3-2d04e7780840 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.934830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ddd0e-aa66-4667-95a3-2d04e7780840 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.935103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ddd0e-aa66-4667-95a3-2d04e7780840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.945154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ddd0e-aa66-4667-95a3-2d04e7780840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:07.946653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ddd0e-aa66-4667-95a3-2d04e7780840 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:07.951296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7274bb-4fab-482a-9e14-1a72c9e44261 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-729ddd0e-aa66-4667-95a3-2d04e7780840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:07.961505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:07.962424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:07.963276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:07.963276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:07.963276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:07.963276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 251/494] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:07.964272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abd5a1-e4e0-4b8e-9948-04df4585a3e3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.964272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abd5a1-e4e0-4b8e-9948-04df4585a3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.964272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abd5a1-e4e0-4b8e-9948-04df4585a3e3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:07.970800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 252/495] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:07.970800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abd5a1-e4e0-4b8e-9948-04df4585a3e3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:07.996962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a32542-55e4-49a8-b190-37ccf4b98120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:07.997615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a32542-55e4-49a8-b190-37ccf4b98120 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:07.997897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a32542-55e4-49a8-b190-37ccf4b98120 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:07.998165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a32542-55e4-49a8-b190-37ccf4b98120 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.007870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a32542-55e4-49a8-b190-37ccf4b98120 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.009397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a32542-55e4-49a8-b190-37ccf4b98120 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-00a32542-55e4-49a8-b190-37ccf4b98120 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:08.023184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:08.024188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:08.025080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 244/496] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:08.025628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f77c8c-bd5b-4b13-afde-2f084d958180 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.026197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f77c8c-bd5b-4b13-afde-2f084d958180 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.026197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f77c8c-bd5b-4b13-afde-2f084d958180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.045670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-61f77c8c-bd5b-4b13-afde-2f084d958180 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:08.046846 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-61f77c8c-bd5b-4b13-afde-2f084d958180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:08.086925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abd5a1-e4e0-4b8e-9948-04df4585a3e3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.117457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 245/497] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:08.118239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f77c8c-bd5b-4b13-afde-2f084d958180 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.122932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.128540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.129506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.129787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.130059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.131036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:08.131492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.133052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.147323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 253/498] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:08.147323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.156404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.226887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f77c8c-bd5b-4b13-afde-2f084d958180 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.251547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33206efd-1d05-4340-aad5-f64509df0fc5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.263028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd408e-2675-4c5c-9431-9350e4bc014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.263803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd408e-2675-4c5c-9431-9350e4bc014a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.264108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd408e-2675-4c5c-9431-9350e4bc014a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.264379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd408e-2675-4c5c-9431-9350e4bc014a None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.265019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd408e-2675-4c5c-9431-9350e4bc014a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:08.265318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd408e-2675-4c5c-9431-9350e4bc014a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.267542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd408e-2675-4c5c-9431-9350e4bc014a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.283614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 246/499] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:08.283614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3114009-0626-4af5-88bf-9b74437d1b95 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.283614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3114009-0626-4af5-88bf-9b74437d1b95 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.283614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3114009-0626-4af5-88bf-9b74437d1b95 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.283614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 247/500] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:08.283614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3114009-0626-4af5-88bf-9b74437d1b95 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.295671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.296306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.296588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.296854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.297897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:08.298192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.299628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.335622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 254/501] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:08.336501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.387071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.393524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.396959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3114009-0626-4af5-88bf-9b74437d1b95 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.431949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba937d-e539-4b47-b31e-bc3c30c42166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.432576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba937d-e539-4b47-b31e-bc3c30c42166 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.432861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba937d-e539-4b47-b31e-bc3c30c42166 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.433134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba937d-e539-4b47-b31e-bc3c30c42166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.441865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91f42f-56c8-45c7-ae9c-c8174ce15bf5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.443225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba937d-e539-4b47-b31e-bc3c30c42166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.444706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba937d-e539-4b47-b31e-bc3c30c42166 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b8ba937d-e539-4b47-b31e-bc3c30c42166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:08.459499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:08.460525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:08.461397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:08.461397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:08.461397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:08.461397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 248/502] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:08.462030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eeb45c-dc8e-4327-8291-d4d6dd8287e7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.462308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eeb45c-dc8e-4327-8291-d4d6dd8287e7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.462579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eeb45c-dc8e-4327-8291-d4d6dd8287e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.480060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.480560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-37eeb45c-dc8e-4327-8291-d4d6dd8287e7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:08.480679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.480953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.481224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.481724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-37eeb45c-dc8e-4327-8291-d4d6dd8287e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:08.488790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.490248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:08.503492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:08.504454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:08.505375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 255/503] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:08.505935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.552709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 249/504] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:08.553745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eeb45c-dc8e-4327-8291-d4d6dd8287e7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.606654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc697b5-9c99-4b2f-ab19-fa124f0558dc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.619998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.635904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.643025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.643569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.643868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.644149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.662669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eeb45c-dc8e-4327-8291-d4d6dd8287e7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.698208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.698838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.699118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.699394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.709954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.711348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.723806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.725443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:08.726449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:08.727515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:08.728557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:08.728557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:08.728557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:08.728557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 250/505] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:08.728795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.834919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1592f99-d176-4c2e-a29d-ac78be66d1db None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.838003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 256/506] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:08.838836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.852395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.861036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:08.868785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.869390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.869763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.870043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.870681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:08.870984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.872388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.884991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 251/507] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:08.885746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:08.938867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ce4d2-2b53-436b-a42b-80db793db93f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.971824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:08.972442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:08.972726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:08.972998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:08.973713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:08.974022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:08.975442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:08.983965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee051b-872b-41f0-a81e-8fcc0c09387a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:08.992979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 257/508] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:08.993906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.020261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.020994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.021280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.021656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.022183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:09.022486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.023864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.035070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 252/509] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:09.035878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.069814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.084585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.099847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c920d-4a37-4416-a57d-16fad99e5270 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.134458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.135079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.135363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.135575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2cdb5-a716-482d-a7ff-8c66b4f910c9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.135669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.145681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:09.150505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:09.151522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 258/510] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:09.152589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.174254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.174879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.175161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.175432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.185804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.187296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:09.201983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:09.203042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:09.204061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:09.204061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:09.204061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:09.207669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 253/511] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:09.208592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.253058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f56f8a-3784-4387-a3cb-435c78ebdcec None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.281399 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.288137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.288766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.289045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.289320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.296623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.298588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.304492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.307504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dffe69-63e3-4255-9917-7ba055e00c48 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:09.310120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:09.311094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 259/512] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:09.312027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.343617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.344925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.345277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.345547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.414648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfadc0-adf5-481f-9e97-ad8fd1c91140 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.416465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.417940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.447627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.448247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.448522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.448793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.449425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.450830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.473121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 260/513] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:51:09 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:09.474310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.527775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.536576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.545753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 254/514] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:09.546786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.590573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1cf46-77c5-47f2-a946-d6dd5eb27bff None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.628483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.629099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.629375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.629646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.630323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:09.630623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.632029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.646731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 261/515] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:09.647531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.662823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9adb-20b4-4b9f-b455-671ec99ad010 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.701436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.701877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.702161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.702436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.703077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:09.703433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.705638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.718110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 255/516] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:09.719604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.774431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fcf75-e903-4b75-ae65-61f167f3057c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.782324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.785337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:09.813469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.814025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.814622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.814622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.815325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:09.815626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.817094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.827476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191445d-9f8c-4184-90c5-1fdcc1086bfb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.835045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 262/517] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:09.835877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.869447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.870067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.870442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.870609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.882882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:09.884741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:09.900672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:09.901753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:09.902754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:09.902754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:09.902754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:09.902754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 256/518] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:09.907769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:09.948861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b32cb-45c0-4ba7-ac1f-ea4ebaa9bf50 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:09.985742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:09.986477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:09.986826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:09.987163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:09.999258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.001521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:10.016471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:10.017588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:10.018439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:10.018439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:10.018439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:10.018439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 263/519] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:10.018767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.031811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c69a-6cd1-45b1-a30f-648d577a3975 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.043124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.044768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.068240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.068867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.069145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.069444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.076111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.077555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:10.093181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:10.094569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:10.095441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 257/520] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:10.095941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.138500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2201df-51be-4bcc-bcbe-70b7d5f6cb00 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.175197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.175805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.176112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.176385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.205305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef58da-1219-4915-9fab-9f744fa1116c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.238834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.239538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.239920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.240257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.240968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:10.241347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.242796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.255870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.256034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 258/521] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:10.256831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.257781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.279060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.296450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.366609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 264/522] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:10.367454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.376511 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452409af-8d6a-4805-863e-88f4219da7e2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.418883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.419563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.419796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.420045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.420682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:10.420983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.422587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.434330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 259/523] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:10.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.481119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bdedb-e127-453e-ae95-5aef041ce5cb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.502046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.521867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.522468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.522745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.523015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.523708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:10.524035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.525869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.542170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.550188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 265/524] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:10.550188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.559044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf1c2c-f237-473e-966a-c7538e4aeafe None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.596352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.596972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.597364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.597639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.604987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.606769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:10.619324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:10.620234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 260/525] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:10.626860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.654989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3843d-72d0-476c-9ee7-f7969457c053 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.692714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.693344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.693629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.693898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.706921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.708402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:10.723012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:10.724182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:10.725242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:10.725242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:10.725242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:10.728663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 266/526] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:10.729357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.736304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b01130-65f6-41ef-8afb-0fbbaea1fee1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.762059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.773098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.774149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.774432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.774706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.784522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.785911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:10.797367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:10.798315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:10.799212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 261/527] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:10.799763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.819020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:10.846045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace043-2a85-43b5-abbb-a3be4f2736f3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.882171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.882779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.883057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.883332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.884198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=9f85882ffc0447baaf3e00554f6b4ed1) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:10.884504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.886047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:10.903868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 267/528] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/role_assignments?user.id=9f85882ffc0447baaf3e00554f6b4ed1&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:10.903868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:10.923151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da793f-ec93-4baf-9f38-4b81006c1908 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:10.965081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:10.966751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:10.967130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:10.967898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:10.969879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:10.970439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:10.973325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.001825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 262/529] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:11.003001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:11.036915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:11.045160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8d9bd-2422-4b41-8ed5-4f78f426843e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:11.089909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:11.090695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.090985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.091262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.091947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:11.092318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:11.095281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.095405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:11.108475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 268/530] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:11.109516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:11.128522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab941a2-e3c4-4f10-9a08-6c0e37a8408b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:11.165998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:11.166977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.167466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.168548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.169847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:11.170352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:11.173619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.221765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 263/531] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Oct 6 10:51:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:11.222631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:11.224652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe97585-1f34-446b-be71-036ae40a3a30 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:11.270392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:11.271378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.271986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.272457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.273453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=91413e6647284487bae08bb7eb02da0b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:11.273960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:11.275873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.291939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 269/532] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:11 2022] GET /identity/v3/role_assignments?user.id=91413e6647284487bae08bb7eb02da0b&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:11.292975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:11.345737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983b63-25ec-4d4f-a7db-32d6e1560f12 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:11.350270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:11.384397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:11.399811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:11.400831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.403025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.403513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.429924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:11.431438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:11.459486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:11.460462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:11.461329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:11.471095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 264/533] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:11.472110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:11.530022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c91d5-139e-426f-ae38-6f3bf6ff699c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:11.576555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6256b5e-03b7-4a27-9130-2804e587b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:11.577186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6256b5e-03b7-4a27-9130-2804e587b4df None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.577472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6256b5e-03b7-4a27-9130-2804e587b4df None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.577748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6256b5e-03b7-4a27-9130-2804e587b4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.666833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26afa61-bcfd-4ce1-942f-116356f45f2d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:11.679801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6256b5e-03b7-4a27-9130-2804e587b4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:11.681441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6256b5e-03b7-4a27-9130-2804e587b4df None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.701548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:11.736793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9d406-ce17-4d5f-8732-7851633cdd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:11.738397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d406-ce17-4d5f-8732-7851633cdd55 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.738775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d406-ce17-4d5f-8732-7851633cdd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.741292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d406-ce17-4d5f-8732-7851633cdd55 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.742136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9d406-ce17-4d5f-8732-7851633cdd55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:11.742549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9d406-ce17-4d5f-8732-7851633cdd55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:11.746168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9d406-ce17-4d5f-8732-7851633cdd55 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.769226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 265/534] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:11.771105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa2dc0-93c1-4a82-92ad-d2e381370bd3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.771397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa2dc0-93c1-4a82-92ad-d2e381370bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.771674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa2dc0-93c1-4a82-92ad-d2e381370bd3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.775959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 266/535] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:11.776852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa2dc0-93c1-4a82-92ad-d2e381370bd3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:11.830400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 270/536] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:11.836511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408630ec-7b2a-4514-b791-2a0357d6b6b4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.836881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408630ec-7b2a-4514-b791-2a0357d6b6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.837265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408630ec-7b2a-4514-b791-2a0357d6b6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.845927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:11.864322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-408630ec-7b2a-4514-b791-2a0357d6b6b4 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:11.865916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-408630ec-7b2a-4514-b791-2a0357d6b6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:11.910263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa2dc0-93c1-4a82-92ad-d2e381370bd3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:11.954902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 271/537] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:11.954902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408630ec-7b2a-4514-b791-2a0357d6b6b4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:11.959172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:11.960017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:11.960620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:11.961137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:11.964614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=6ecccd27dfa04cf985c9fd632ce2fe60) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:11.965149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:11.966765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:11.978374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 267/538] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:11 2022] GET /identity/v3/role_assignments?user.id=6ecccd27dfa04cf985c9fd632ce2fe60&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:12.021654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:12.091447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408630ec-7b2a-4514-b791-2a0357d6b6b4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:12.128758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:12.129409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:12.129685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:12.129960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:12.141282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:12.142750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:12.159191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:12.160287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:12.161162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:12.161162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:12.161162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:12.161162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 272/539] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:12.167581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:12.314905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141ba85-cd07-438e-84fa-14adff6fcf4e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:12.315729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:12.346628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:12.395320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:12.396596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:12.397085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:12.397562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:12.398437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:12.400154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:12.401759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:12.424517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 268/540] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:12.432893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:12.513133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:12.521943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c109db-3bd5-4b7e-ae40-ba8d714c6b86 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:12.568573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fce494-d373-4f91-9797-eb1aa012b99c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:12.595095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:12.595755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:12.596024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:12.596297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:12.612744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:12.618013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:12.618922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:12.619726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 273/541] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:12.626837 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:12.631112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:12.631756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:12.632061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:12.632335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:12.646811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:12.648445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:12.674521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:12.675446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:12.676318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 269/542] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:12.689908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:12.735653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5419b99-1660-4e9d-bd34-4e08463572c2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:12.758666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:12.790982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:12.791594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:12.791884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:12.792159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:12.792825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:12.793124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:12.794480 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:12.814198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 274/543] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:12.820091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:12.864317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:12.896131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85135210-0e07-4b4b-af79-20202adffc37 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:12.969561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:12.970220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:12.970500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:12.970767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:12.971454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:12.971755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:12.973226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:12.996018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 270/544] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:13.003071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:13.019050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31697d5c-b96f-43be-8a40-ffcaa1dd6c59 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:13.088858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:13.089844 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.090367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.090860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.121274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:13.123084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:13.154778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:13.156062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:13.157471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:13.157471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:13.157471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:13.163672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 275/545] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:13.169814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:13.179940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:13.193969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ce2c0-1065-4fc5-8525-8e556ed58786 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:13.266805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fa684-d9e5-4b44-b46c-eb9f072acdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:13.267587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa684-d9e5-4b44-b46c-eb9f072acdd5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.267879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa684-d9e5-4b44-b46c-eb9f072acdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.268150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa684-d9e5-4b44-b46c-eb9f072acdd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.268866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa684-d9e5-4b44-b46c-eb9f072acdd5 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=b5c892640b3e409b9a99b4ca1d34a964) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:13.269165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa684-d9e5-4b44-b46c-eb9f072acdd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:13.274565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa684-d9e5-4b44-b46c-eb9f072acdd5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:13.286660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 271/546] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:13 2022] GET /identity/v3/role_assignments?user.id=b5c892640b3e409b9a99b4ca1d34a964&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:13.316496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:13.330401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ebab-5e29-4a53-b8f6-4a502f9e7ee6 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.330727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ebab-5e29-4a53-b8f6-4a502f9e7ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.330989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ebab-5e29-4a53-b8f6-4a502f9e7ee6 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.335979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 272/547] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:13.338374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a9ec2-200c-4d98-9ff3-a95a5de4d793 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.338674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a9ec2-200c-4d98-9ff3-a95a5de4d793 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.338983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a9ec2-200c-4d98-9ff3-a95a5de4d793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.358028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-477a9ec2-200c-4d98-9ff3-a95a5de4d793 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:13.359262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-477a9ec2-200c-4d98-9ff3-a95a5de4d793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:13.368900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a3b6f-2264-45f2-969e-5d2afcedd263 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:13.417522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:13.418148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.418501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.418872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.419988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:13.420562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:13.422074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:13.448615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 276/548] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:13.460065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:13.478324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 273/549] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:51:13.622653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39c69b-6a83-4e42-91d2-0c0903e20027 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:13.674526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984ddda-78d9-41fd-b370-c6b73899df57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:13.675231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984ddda-78d9-41fd-b370-c6b73899df57 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.675511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984ddda-78d9-41fd-b370-c6b73899df57 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.675779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984ddda-78d9-41fd-b370-c6b73899df57 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.681600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:13.688806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984ddda-78d9-41fd-b370-c6b73899df57 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:13.690325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984ddda-78d9-41fd-b370-c6b73899df57 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9984ddda-78d9-41fd-b370-c6b73899df57 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:13.709037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:13.710775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:13.711785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 277/550] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:13.717106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a9ec2-200c-4d98-9ff3-a95a5de4d793 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:13.764555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59168a1d-05ae-424b-87fb-a4036d517217 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.764894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59168a1d-05ae-424b-87fb-a4036d517217 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.765097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59168a1d-05ae-424b-87fb-a4036d517217 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.767348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 278/551] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:13.772676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b0b3-b495-4c94-9243-f39252d15836 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.772950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b0b3-b495-4c94-9243-f39252d15836 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.773223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b0b3-b495-4c94-9243-f39252d15836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.800811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4fc1b0b3-b495-4c94-9243-f39252d15836 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:13.801997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4fc1b0b3-b495-4c94-9243-f39252d15836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:13.846630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:13.862289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a9ec2-200c-4d98-9ff3-a95a5de4d793 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:13.908600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:13.909125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:13.909404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:13.909673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:13.910394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:13.910510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 279/552] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:13.910691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:13.911477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b0b3-b495-4c94-9243-f39252d15836 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:13.912156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:13.952312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 274/553] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Oct 6 10:51:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:13.959582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.020617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b0b3-b495-4c94-9243-f39252d15836 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.050438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:14.063047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.063677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.063995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.064270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.075084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:14.080094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:14.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:14.081944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 280/554] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:14.087527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.142137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f578a2c-4ffc-4a90-a2d4-5cff2f109421 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.202908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95897547-0cf7-4ed4-b37a-bdd90f7c4338 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.218215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.218866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.219248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.219519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.255078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:14.267022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.267697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.268006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.268289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.268956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:14.269257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:14.270880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:14.280203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 281/555] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:14.286758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.340295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:14.341001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:14.342409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:14.409239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4c169-41ba-4de2-b261-add127f4d70d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.446959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.447594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.447889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.448165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.460864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:14.462241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:14.477938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 275/556] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:14.478590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:14.479484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:14.480387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:14.480387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:14.480387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:14.481441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.485129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 282/557] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:14.486732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.546507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:14.607249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370cc7e-f5b9-44f3-9663-d3e231d9a2ba None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.611418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:14.616573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f0bc-c841-4a5b-8ea0-1a0c1bfbf713 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.646930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.647687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.647998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.648269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.660824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.661420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:14.662276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.662759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.663591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:14.666158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:14.667079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 276/558] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:14.668342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.679931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:14.681941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:14.700831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:14.701891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:14.702880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:14.702880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:14.702880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:14.705616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 283/559] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:14.706848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.793171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b43d7-6290-469f-93aa-1559e4a170d5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.802579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:14.831538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.832228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.832506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.832784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.833415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:14.833735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:14.835086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:14.839449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709e509-4f5f-4657-8515-86e66b4fbf47 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.846503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 277/560] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:14.847297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.879737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527adc1-3754-46cc-b7cd-b1ce6e52a5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:14.880564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527adc1-3754-46cc-b7cd-b1ce6e52a5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.880721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527adc1-3754-46cc-b7cd-b1ce6e52a5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.880886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527adc1-3754-46cc-b7cd-b1ce6e52a5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.881535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527adc1-3754-46cc-b7cd-b1ce6e52a5b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:14.881836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527adc1-3754-46cc-b7cd-b1ce6e52a5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:14.883189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527adc1-3754-46cc-b7cd-b1ce6e52a5b9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:14.895874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 284/561] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:14.895874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54abbb1-504c-4a58-be38-7b1995e9dd27 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:14.895874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54abbb1-504c-4a58-be38-7b1995e9dd27 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:14.895874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54abbb1-504c-4a58-be38-7b1995e9dd27 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:14.895874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 285/562] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:14.895874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54abbb1-504c-4a58-be38-7b1995e9dd27 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:14.911788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:14.962543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c38f82-1244-4538-9f14-89981d60a7ad None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:14.999227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.000071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.000803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.001051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.001886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:15.002099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.003470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.005291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54abbb1-504c-4a58-be38-7b1995e9dd27 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.020724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 278/563] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:15.020724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.045823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb724443-ac72-497b-a8c6-2a40f4c5ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.046398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb724443-ac72-497b-a8c6-2a40f4c5ab4f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.046549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:15.046697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb724443-ac72-497b-a8c6-2a40f4c5ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.047579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb724443-ac72-497b-a8c6-2a40f4c5ab4f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.057772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb724443-ac72-497b-a8c6-2a40f4c5ab4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.059237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb724443-ac72-497b-a8c6-2a40f4c5ab4f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-eb724443-ac72-497b-a8c6-2a40f4c5ab4f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:15.073572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:15.074551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 10:51:15.075492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:15.081850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 286/564] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:15.081850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8803c4-535e-4abf-bbae-8b389c0ec969 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.081850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8803c4-535e-4abf-bbae-8b389c0ec969 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.081850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8803c4-535e-4abf-bbae-8b389c0ec969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.097646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2d8803c4-535e-4abf-bbae-8b389c0ec969 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:15.098856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2d8803c4-535e-4abf-bbae-8b389c0ec969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:15.130906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf918a5d-bceb-4170-8c94-89d6715e2932 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.170460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.171457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.172010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.172131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 287/565] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:15.172597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.173263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8803c4-535e-4abf-bbae-8b389c0ec969 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.175549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:15.188029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.189751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:15.205753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:15.206755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:15.207635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:15.207635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:15.207635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:15.209701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 279/566] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:15.210645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.287974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8803c4-535e-4abf-bbae-8b389c0ec969 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.304169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:15.332418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc4a71-1f71-4c7b-95b8-5375c544369c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.336464 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.337102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.337383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.337652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.347249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.349014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:15.366432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:15.367366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:15.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 288/567] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:15.369209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.374251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.375674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.376257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.376734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.377654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:15.378152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.379851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.414765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 280/568] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:15.415992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.445632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:15.499825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c9f32-d80f-4e56-86fc-c6ffbf2971d1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.533576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff380b15-d6ab-4727-8e7e-1d69addf79e5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.542705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.543332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.543611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.543909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.544604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:15.544910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.546310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.564476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 289/569] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:15.564476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.574960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.575656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.575980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.576263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.576934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:15.577232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.578990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.581516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:15.598142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 281/570] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:15.598142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.670425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568ab41-63d2-461e-8782-7de12c128939 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.693536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:15.698544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71ca49-6f17-4040-a663-bdba2ebfaee0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.706033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.706654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.706930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.707204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.716914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.718360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.732502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.733118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.733393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.733658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:15.739193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:15.740321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:15.741347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:15.741347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:15.741347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:15.741347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 290/571] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:15.741347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.746721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:15.752184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:15.753290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 282/572] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:15.754139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.815448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:15.845307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b046e-bd8a-4d4a-9ad9-ff286caf0560 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.859648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3e1d3-2326-4dd1-90ce-1fd320b56e34 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:15.877201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf844f3a-b7f4-4fa6-998f-491e48971b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.877821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf844f3a-b7f4-4fa6-998f-491e48971b14 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.878097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf844f3a-b7f4-4fa6-998f-491e48971b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.878372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf844f3a-b7f4-4fa6-998f-491e48971b14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.885542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf844f3a-b7f4-4fa6-998f-491e48971b14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.886959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf844f3a-b7f4-4fa6-998f-491e48971b14 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.895234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:15.895820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.896121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.896388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.897093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=df5aa06f9803465bb102482d351adeac) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:15.897395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:15.898808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bf844f3a-b7f4-4fa6-998f-491e48971b14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:15.898936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:15.899905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:15.900911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 291/573] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:15.902130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c0347-48e3-423a-8b72-97f900cab48c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:15.902130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c0347-48e3-423a-8b72-97f900cab48c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:15.902130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c0347-48e3-423a-8b72-97f900cab48c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:15.903333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 292/574] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:15.908150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c0347-48e3-423a-8b72-97f900cab48c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.908311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 283/575] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/role_assignments?user.id=df5aa06f9803465bb102482d351adeac&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:15.908311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:15.959260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.028226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d551e05-0541-4c74-94ef-fe09c4d22b76 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.047391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c0347-48e3-423a-8b72-97f900cab48c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.057720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.075299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.075867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.076651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.076651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.077179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:16.077418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.078859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.089243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 284/576] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:16.090079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.097067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc65b2a-009d-4f46-80cb-574e1bb6b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.099008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc65b2a-009d-4f46-80cb-574e1bb6b81e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.099474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc65b2a-009d-4f46-80cb-574e1bb6b81e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.099749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc65b2a-009d-4f46-80cb-574e1bb6b81e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.100438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc65b2a-009d-4f46-80cb-574e1bb6b81e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:16.100751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc65b2a-009d-4f46-80cb-574e1bb6b81e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.102189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc65b2a-009d-4f46-80cb-574e1bb6b81e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.121044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 293/577] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:16.121044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb4b22-6fb1-4599-8723-cced832efd88 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.121044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb4b22-6fb1-4599-8723-cced832efd88 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.121044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb4b22-6fb1-4599-8723-cced832efd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.137875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dfbb4b22-6fb1-4599-8723-cced832efd88 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:16.139063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dfbb4b22-6fb1-4599-8723-cced832efd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:16.211961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673e1bb-07d5-40c9-9aea-64c3301bf535 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.215014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 294/578] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:16.215987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb4b22-6fb1-4599-8723-cced832efd88 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.241750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.271527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.273162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.273682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.274430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.275651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:16.277143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.279564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.304744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 285/579] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:16.306429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.326250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb4b22-6fb1-4599-8723-cced832efd88 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.358889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.378138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.378949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.379561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.379873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.391930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.393493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:16.415546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:16.416632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:16.417534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:16.417534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:16.417534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:16.417534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 295/580] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:16.417534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.450449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83f1ea-7568-491d-be15-93ad1b0ca5e5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.494494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.497527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.498320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.498748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.499088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.507694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.509314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.519636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672596e7-47a2-4744-97b8-1c6dda2b7c5e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:16.521271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:16.522229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 286/581] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:16.524342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.558933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.559676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.560682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.560682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.572692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.574148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:16.590524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:16.591475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:16.592435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:16.592435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:16.592435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:16.592435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 296/582] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:16.592671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.639603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.642807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9052a-e9d4-4277-a203-70ea74e8bf1a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.678704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.679316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.679598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.679903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.680546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:16.680850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.682213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.694696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 287/583] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:16.695507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.712158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fedfef-30a2-4b75-a285-9150c0ec29f8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.742413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.752745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.753703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.754168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.754686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.766694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:16.773194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:16.774130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:16.774987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 297/584] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:16.776383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.801482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff2ca-5e8d-4a98-84df-1108fb05454a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.842616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.843584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.844870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.845574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.847055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:16.847829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.850715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.865063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 288/585] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:16.866059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.888297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979579e-596f-4071-b6a5-be6f03f2c8df None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.893174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:16.926042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:16.926649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:16.926933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:16.927211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:16.927871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:16.928186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:16.929552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:16.942637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 298/586] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:16.947830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:16.979484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef447051-b2f9-4082-8ba0-078d6ece5f80 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:16.998697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:17.015281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.016527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.016816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.017089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.025187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.026650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:17.038325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:17.039276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 289/587] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:17.040269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.060419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0785f8-47fb-41f0-9580-eade60ac4dfd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.098477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59763433-14b1-4002-927f-320c396cfebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.099128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59763433-14b1-4002-927f-320c396cfebc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.099410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59763433-14b1-4002-927f-320c396cfebc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.099827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59763433-14b1-4002-927f-320c396cfebc None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.100487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59763433-14b1-4002-927f-320c396cfebc None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:17.100792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59763433-14b1-4002-927f-320c396cfebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.102386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59763433-14b1-4002-927f-320c396cfebc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.110994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 299/588] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:17.113238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59763433-14b1-4002-927f-320c396cfebc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.126827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:17.151529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecb5e2-5c08-4921-a3b0-e088f5e0b0e7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.192008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.193466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.193753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.194027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/9f85882ffc0447baaf3e00554f6b4ed1/roles/7565190481584409b1756297cec264cd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.211697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=9f85882ffc0447baaf3e00554f6b4ed1, role_id=7565190481584409b1756297cec264cd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.213593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.224080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59763433-14b1-4002-927f-320c396cfebc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.239876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 290/589] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:17 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/9f85882ffc0447baaf3e00554f6b4ed1/roles/7565190481584409b1756297cec264cd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:51:17.239876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.245217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:17.274498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.275285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.275644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.276034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.285563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.287219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:17.302389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:17.303346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:17.304263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 300/590] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:17.305402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.366909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0368647-fe94-4550-a1e8-ad47a6a13682 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.404487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:17.414352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.415282 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.415568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.415860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.416796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:17.417106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.419503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.445324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 291/591] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:17.446186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.460424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b7721-c280-44ac-b692-e565e0df4b36 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.504279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.504910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.505191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.505466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.519129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.520894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:17.538525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:17.539534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:17.540459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:17.540459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:17.540459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:17.544216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 301/592] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:17.545125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.572071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c9950-d9f6-4dd0-9a22-7291856e558f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.576798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:17.621168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.622617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.623056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.623481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.624331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:17.625368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.627936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.644543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 292/593] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:17.646030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.672872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9ff8e-6fa9-4c23-b0fe-ec208faca2ca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.710772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:17.715152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.716800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.717278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.717659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.732425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.735072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:17.752863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:17.753881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:17.754780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:17.754780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:17.754780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:17.754780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 302/594] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:17.755878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.778873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93796552-7390-42ac-b846-a0bcb540df10 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.826327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.827495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.828634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.829397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.830903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:17.831388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.832974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.844219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:17.857125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 293/595] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:17.858079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.876166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03cfe-3494-4404-82a1-efc686fa3238 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:17.916330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:17.917229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:17.917679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:17.918093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/91413e6647284487bae08bb7eb02da0b/roles/7565190481584409b1756297cec264cd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:17.937657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=91413e6647284487bae08bb7eb02da0b, role_id=7565190481584409b1756297cec264cd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:17.939365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:17.961499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 303/596] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:17 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/91413e6647284487bae08bb7eb02da0b/roles/7565190481584409b1756297cec264cd => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:51:17.962585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:17.981906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49b1f4-a252-4fa4-96a4-555de9f56c80 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.001528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:18.052724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.053512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.053898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.054336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.055147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:18.055555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.057242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.082642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 294/597] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:18.088150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:18.097115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63f01c-4f55-4667-aed7-fa2bab308647 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.132454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:18.147057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb22645-5678-4ba8-933b-d6d90d750064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.147716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb22645-5678-4ba8-933b-d6d90d750064 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.148086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb22645-5678-4ba8-933b-d6d90d750064 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.148365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb22645-5678-4ba8-933b-d6d90d750064 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.161476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb22645-5678-4ba8-933b-d6d90d750064 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.163225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb22645-5678-4ba8-933b-d6d90d750064 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7fb22645-5678-4ba8-933b-d6d90d750064 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:18.178875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:18.179924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 10:51:18.180900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:18.183885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 304/598] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:18.185495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242ca0-d2ff-4d86-abbd-d474ebcaf14d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.185922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242ca0-d2ff-4d86-abbd-d474ebcaf14d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.186393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242ca0-d2ff-4d86-abbd-d474ebcaf14d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.188130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 305/599] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:18.189223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242ca0-d2ff-4d86-abbd-d474ebcaf14d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:18.237097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63b2f8-25cc-4030-ad09-0e8e96287360 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.285715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406709fc-824c-4019-913d-9165fdc8cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.286717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406709fc-824c-4019-913d-9165fdc8cb25 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.287359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406709fc-824c-4019-913d-9165fdc8cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.288186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406709fc-824c-4019-913d-9165fdc8cb25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.297697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406709fc-824c-4019-913d-9165fdc8cb25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.299431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406709fc-824c-4019-913d-9165fdc8cb25 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-406709fc-824c-4019-913d-9165fdc8cb25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:18.312289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:18.313358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:18.314272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 295/600] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:18.315232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d937720-2025-4457-ab90-263cb544538d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.315341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d937720-2025-4457-ab90-263cb544538d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.315574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d937720-2025-4457-ab90-263cb544538d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.334978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9d937720-2025-4457-ab90-263cb544538d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:18.336214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9d937720-2025-4457-ab90-263cb544538d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:18.337511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242ca0-d2ff-4d86-abbd-d474ebcaf14d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.364192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:18.376294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.377175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.377658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.378283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.379231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:18.379737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.381387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.403005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 306/601] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Oct 6 10:51:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:18.404167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:18.415289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 296/602] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:18.419933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d937720-2025-4457-ab90-263cb544538d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:18.492954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:18.596492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d937720-2025-4457-ab90-263cb544538d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.606664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b1e13-0cf6-429e-ba40-a44fc001a31c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.664268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.664804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.665092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.665369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.669351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.669961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.670242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.670516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.671212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:18.671519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.673645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.689134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.690595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.698228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 307/603] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:18.704625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:18.717318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:18.718328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:18.719805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:18.719805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:18.719805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:18.719805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 297/604] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:18.726051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:18.748480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:18.836725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba021b-89c1-4f9c-86cf-dccb93eb3069 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.838301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:18.865470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f059862-f4fa-4a78-b523-43cb25d7d6ff None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:18.881123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.881644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.881931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.882211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/6ecccd27dfa04cf985c9fd632ce2fe60/roles/7565190481584409b1756297cec264cd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.900495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=6ecccd27dfa04cf985c9fd632ce2fe60, role_id=7565190481584409b1756297cec264cd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.902206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.918274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:18.919337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:18.919820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:18.920405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:18.921267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:18.921588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 308/605] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:18 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/6ecccd27dfa04cf985c9fd632ce2fe60/roles/7565190481584409b1756297cec264cd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:51:18.922563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:18.923390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:18.924648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:18.940378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 298/606] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:18.953594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:19.065053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1bf05-418f-4355-8517-46709f0f77ad None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:19.106545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:19.107194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.107478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.107753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.117188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:19.122213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:19.128688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:19.129750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:19.130594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 309/607] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:19.137248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:19.156179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:19.194980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc85e1d-530d-47ad-b401-1b936a5f0d2d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:19.236222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:19.237399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.237894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.238374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.246609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:19.248612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:19.256347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236baac3-ee6a-40fc-a6a4-8196ba9f2484 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:19.262473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:19.263411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:19.264413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:19.267612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 299/608] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:19.277121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:19.303040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:19.303659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.303992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.304323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.305001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:19.305357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:19.306729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:19.322672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 310/609] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:19.326735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:19.466797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b147-9c72-4b26-b6b9-e39aa28f121a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:19.467713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:19.500919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888caaf-5ac2-4fc4-8799-3cf5076a7333 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:19.529331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:19.543218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:19.543867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.544139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.544408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.545107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:19.545409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:19.547408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:19.574259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 300/610] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:19.575756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:19.576416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.576694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.576962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.578888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:19.600388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:19.601853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:19.617217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:19.618203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:19.619090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:19.619090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:19.619090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:19.619090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 311/611] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:19.625957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:19.720530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423b0d6-fc06-4adf-856d-f033b3a462c9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:19.748487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff3bda-904c-4ea3-aa39-937a1664dd2f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:19.757111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173bdb6-f489-4906-9387-e033c79ebafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:19.757769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173bdb6-f489-4906-9387-e033c79ebafb None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.758044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173bdb6-f489-4906-9387-e033c79ebafb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.758316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173bdb6-f489-4906-9387-e033c79ebafb None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/b5c892640b3e409b9a99b4ca1d34a964/roles/7565190481584409b1756297cec264cd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.775051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173bdb6-f489-4906-9387-e033c79ebafb None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=b5c892640b3e409b9a99b4ca1d34a964, role_id=7565190481584409b1756297cec264cd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:19.776518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173bdb6-f489-4906-9387-e033c79ebafb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:19.780526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:19.794111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:19.794399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 301/612] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:19 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/b5c892640b3e409b9a99b4ca1d34a964/roles/7565190481584409b1756297cec264cd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:51:19.795769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd60c5-8cca-4072-a4bc-562e73085c20 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.796173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd60c5-8cca-4072-a4bc-562e73085c20 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.796545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd60c5-8cca-4072-a4bc-562e73085c20 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.798868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 302/613] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:19.799583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.807873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.807873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.807873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:19.807873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:19.808271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ff56-55b0-4ae9-891c-a9ebc96c617b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:19.808271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ff56-55b0-4ae9-891c-a9ebc96c617b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:19.808271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ff56-55b0-4ae9-891c-a9ebc96c617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:19.808472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:19.831656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 312/614] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:19.836307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:19.838075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7464ff56-55b0-4ae9-891c-a9ebc96c617b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:19.842592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7464ff56-55b0-4ae9-891c-a9ebc96c617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:19.918220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:19.963876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 303/615] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:51:20.045110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb3de-f1c5-4353-9ac1-548ebc193901 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:20.084939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:20.085941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.086482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.086977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.094737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:20.096471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:20.108513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:20.109565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:20.110680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:20.113196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 313/616] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:20.118430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ff56-55b0-4ae9-891c-a9ebc96c617b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:20.241112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ff56-55b0-4ae9-891c-a9ebc96c617b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:20.275545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:20.304244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:20.304779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.305055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.305333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.306098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:20.306452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:20.307953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:20.338550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:20.339776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 304/617] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Oct 6 10:51:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:20.345855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:20.408711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6448-2358-4897-8041-9efe6e581b9c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:20.413211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:20.463907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d7f83-6225-4fb7-8624-511183d6fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:20.464778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7f83-6225-4fb7-8624-511183d6fb93 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.465208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7f83-6225-4fb7-8624-511183d6fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.465637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7f83-6225-4fb7-8624-511183d6fb93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.480547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d7f83-6225-4fb7-8624-511183d6fb93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f43d7f83-6225-4fb7-8624-511183d6fb93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:20.488306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:20.489295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:20.490146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 314/618] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:20.496337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafb2fb-43a7-4dd3-8c3d-604784661955 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.498413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafb2fb-43a7-4dd3-8c3d-604784661955 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.498877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafb2fb-43a7-4dd3-8c3d-604784661955 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.500954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 315/619] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:20.501775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafb2fb-43a7-4dd3-8c3d-604784661955 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:20.526157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c309-0ad1-4226-bb9f-d8ac092a1cab None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:20.566262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13af0f-71fd-42af-a190-a9c2e8848a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:20.567331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13af0f-71fd-42af-a190-a9c2e8848a33 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.567817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13af0f-71fd-42af-a190-a9c2e8848a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.568316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13af0f-71fd-42af-a190-a9c2e8848a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.580616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13af0f-71fd-42af-a190-a9c2e8848a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:20.582482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13af0f-71fd-42af-a190-a9c2e8848a33 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-9d13af0f-71fd-42af-a190-a9c2e8848a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:20.599872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:20.600882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:20.601792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:20.601792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:20.601792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:20.604099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 305/620] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:20.605649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6640d-11e1-4d05-b95f-88da24d7ff46 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.606190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6640d-11e1-4d05-b95f-88da24d7ff46 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.606664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6640d-11e1-4d05-b95f-88da24d7ff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.618093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafb2fb-43a7-4dd3-8c3d-604784661955 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:20.626006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:20.630251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-daa6640d-11e1-4d05-b95f-88da24d7ff46 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:20.631805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-daa6640d-11e1-4d05-b95f-88da24d7ff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:20.656289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:20.657088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.657438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.657806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.658642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:20.659014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:20.660596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:20.669628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 316/621] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:20.670634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:20.712802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 306/622] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:20.713752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6640d-11e1-4d05-b95f-88da24d7ff46 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:20.728692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:20.818837 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3aeb9-1861-470c-ac5a-2ad1cdf3af46 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:20.888011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:20.888625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.888908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.889184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.889831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:20.890137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:20.891582 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:20.918448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6640d-11e1-4d05-b95f-88da24d7ff46 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:20.928131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 317/623] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:20.933679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:20.965183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:20.965807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:20.966090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:20.966366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:20.988381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:20.989863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:20.995295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:21.017837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:21.018855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:21.019740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:21.019740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:21.019740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:21.020354 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 307/624] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:21.021271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.075105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a28bfd-44f8-48dc-8ef2-72108d365dd5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.113344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.114765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.115643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.116070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.116449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.123781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.125606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:21.137435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:21.138568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:21.139679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:21.146626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 318/625] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:21.147572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.171386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9181433-411d-4108-b920-ebf98ec1eaaa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.209650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.210692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.210957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.211337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.224622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:21.229941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:21.230853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:21.231725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 308/626] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:21.234237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.271895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.287402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042be9af-7b2e-43bd-8bed-e9acb20a36cb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.328438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.329543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.329921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.330304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.331328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:21.331711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.333177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.342772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18593b7-2a95-4b1b-85a2-9a3a99d888b4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.348596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 319/627] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:21.349641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.374458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.388531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.395901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.395901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.395901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.395901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:21.395901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.395901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.410460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 309/628] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:21.410460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.479004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6180ad-25da-49d8-a839-e671f9d1ae25 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.518833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.519750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.520253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.520630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.521473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:21.521885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.523637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.531361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744f62c-8922-4b28-aee3-fa01eda07a17 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.534655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 320/629] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:21.535660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.550527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.577290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.578941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.579268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.579912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.587434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.589149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:21.601899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:21.603073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 10:51:21.604294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:21.611627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 310/630] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:21.612725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.660573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.667288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804cde-5270-4290-850c-c0b9facd93ca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.704510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.705173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.705457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.705736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/df5aa06f9803465bb102482d351adeac/roles/7565190481584409b1756297cec264cd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.721137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2899-98f7-4c3e-a800-6646b3a7ced2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.722139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=df5aa06f9803465bb102482d351adeac, role_id=7565190481584409b1756297cec264cd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.724589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.738463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 321/631] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:21 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/df5aa06f9803465bb102482d351adeac/roles/7565190481584409b1756297cec264cd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:51:21.739608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.769922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bd6c5-42bd-4ea4-98f5-5659c7878b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.770591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bd6c5-42bd-4ea4-98f5-5659c7878b9e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.771027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bd6c5-42bd-4ea4-98f5-5659c7878b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.771530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bd6c5-42bd-4ea4-98f5-5659c7878b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.787874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bd6c5-42bd-4ea4-98f5-5659c7878b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.789779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bd6c5-42bd-4ea4-98f5-5659c7878b9e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-913bd6c5-42bd-4ea4-98f5-5659c7878b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:21.810412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:21.811429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 311/632] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578dfc-12bb-451e-aeef-042bfac11297 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578dfc-12bb-451e-aeef-042bfac11297 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578dfc-12bb-451e-aeef-042bfac11297 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.816883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 312/633] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:21.825608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578dfc-12bb-451e-aeef-042bfac11297 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:21.842064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.892127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4da18a-5bb0-4761-9f27-3ed8c6d432e7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:21.932853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e7df3-ee17-4e86-a06f-8502b5c1336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:21.933473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e7df3-ee17-4e86-a06f-8502b5c1336f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.933751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e7df3-ee17-4e86-a06f-8502b5c1336f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.934021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e7df3-ee17-4e86-a06f-8502b5c1336f None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.934823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e7df3-ee17-4e86-a06f-8502b5c1336f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:21.935106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e7df3-ee17-4e86-a06f-8502b5c1336f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:21.936630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e7df3-ee17-4e86-a06f-8502b5c1336f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:21.958111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 322/634] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:21.959587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0b9c-1422-4071-bb79-b7bac447e48b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:21.959956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0b9c-1422-4071-bb79-b7bac447e48b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:21.960296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0b9c-1422-4071-bb79-b7bac447e48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:21.973566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:21.982199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b89d0b9c-1422-4071-bb79-b7bac447e48b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:21.983477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b89d0b9c-1422-4071-bb79-b7bac447e48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:21.985098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578dfc-12bb-451e-aeef-042bfac11297 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.026425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.027373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.027826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.028848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.029965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:22.030455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:22.033771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:22.051287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 313/635] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:22.052695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.057992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 323/636] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:22.058778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0b9c-1422-4071-bb79-b7bac447e48b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.125799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:22.183233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef34ab-32e7-4a39-8364-b7442da1b83a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.203424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0b9c-1422-4071-bb79-b7bac447e48b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.218459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.219076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.219351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.219625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.248225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:22.270445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.271390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.271827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.272356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.289205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:22.291143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:22.294216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:22.295747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:22.314426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:22.315641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:22.316868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:22.325797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 324/637] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:22.340469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.415173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 314/638] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:22.420870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.485126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d6f4-9a9f-473b-96b3-1e143f224a79 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.491004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:22.497131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:22.525669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdd343-e181-4ec5-90fe-e4b2240aab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.526386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdd343-e181-4ec5-90fe-e4b2240aab09 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.526667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdd343-e181-4ec5-90fe-e4b2240aab09 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.527068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdd343-e181-4ec5-90fe-e4b2240aab09 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.527776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdd343-e181-4ec5-90fe-e4b2240aab09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:22.528160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdd343-e181-4ec5-90fe-e4b2240aab09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:22.529699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdd343-e181-4ec5-90fe-e4b2240aab09 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:22.542977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280b6e9-e0d6-4f5d-ba57-c6d686dc1144 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.553476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 325/639] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:22.553476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6232627-29cf-4148-90a9-4aa7526bdaa9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.553476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6232627-29cf-4148-90a9-4aa7526bdaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.553476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6232627-29cf-4148-90a9-4aa7526bdaa9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.553476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 326/640] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:22.553476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6232627-29cf-4148-90a9-4aa7526bdaa9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.586059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.586916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.587342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.587758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.598937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:22.600652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:22.615791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:22.616785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:22.617706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:22.617706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:22.617706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:22.617706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 315/641] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:22.617706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.665484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6232627-29cf-4148-90a9-4aa7526bdaa9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.702787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3856f09-354c-477f-b0fd-809ba2b291bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.704031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3856f09-354c-477f-b0fd-809ba2b291bf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.704500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3856f09-354c-477f-b0fd-809ba2b291bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.704780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3856f09-354c-477f-b0fd-809ba2b291bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.715142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3856f09-354c-477f-b0fd-809ba2b291bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f3856f09-354c-477f-b0fd-809ba2b291bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:22.719944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:22.720973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:22.721849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 327/642] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:22.722556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831fcfb-99c2-4d56-b435-313e15e12081 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.722556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831fcfb-99c2-4d56-b435-313e15e12081 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.723281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831fcfb-99c2-4d56-b435-313e15e12081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.727875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db75e9-fb37-4e9b-b6fd-91fbd94113c0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.737971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:22.741099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8831fcfb-99c2-4d56-b435-313e15e12081 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:22.742274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8831fcfb-99c2-4d56-b435-313e15e12081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:22.765972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:22.772529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.773138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.773419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.773694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.823985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 328/643] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:22.825287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831fcfb-99c2-4d56-b435-313e15e12081 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.853992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:22.856885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:22.944785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831fcfb-99c2-4d56-b435-313e15e12081 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:22.982610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:22.983489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:22.984595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:22.985066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:22.985922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:22.986106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 316/644] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:22.986547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:22.987124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:22.988258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:22.996416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.003698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 329/645] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:23.008606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.032130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.103166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334cbbd-ea68-40f8-8c5a-cde010b1ae59 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.128825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740acc1d-5a71-4ce0-959c-b833deeeb835 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.141677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.142208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.142483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.142754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.143411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:23.143710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.145114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.153215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 317/646] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:23.154004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.171594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.173224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.173515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.173778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.184332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.185773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:23.201323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:23.202288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:23.204071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:23.204071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:23.204071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:23.204071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 330/647] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:23.204071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.227694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.262242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e021e7-cb00-48ae-ab58-f664a9b80f80 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.275483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.301143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.301774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.302052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.302320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.309005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.310440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.314583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602913-8f0d-4cba-a36c-3e629183683a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:23.322705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:23.323603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:23.324516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 318/648] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:23.325003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.355978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.356680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.356959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.357256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.368393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:23.373679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:23.374651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:23.375696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 331/649] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:23.376981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.441689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51865a24-6cac-40c1-b562-763031ef0326 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.470083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.478869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.479641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.480050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.480394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.485374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263f45-ad99-4e1a-af5b-19a8e92a1c7b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.491812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.493367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:23.507924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:23.509093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:23.514874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:23.514874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:23.514874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:23.514874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 319/650] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:23.514874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.521459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.530005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.530708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.530921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.531692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.532372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:23.532669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.534365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.550580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 332/651] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:23.551443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.623325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74171f6-da93-4704-9364-44f18dbf045f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.654415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab228d-eb7d-4285-a82f-17c0507b1fb3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.660487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.661094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.661369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.661637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.662316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:23.663028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.664509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.678719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 320/652] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:23.679671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.696418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.697047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.697315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.697590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.698257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:23.698562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.699949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.709084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 333/653] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:23.709084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.730143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.767799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.801239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6c7e9-e35c-410b-a93b-f2eeb79f11dc None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.831588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861d6e-0a92-4438-95dd-9b1b6dd28db1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:23.842666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.843785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.844304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.844634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.845280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:23.845585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.846959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.859920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 321/654] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:23.860775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.868227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:23.869170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:23.869794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:23.870306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:23.877902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:23.880328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:23.893021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:23.893923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:23.894823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:23.897103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 334/655] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:23.897919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:23.982903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:23.988204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a752a4-831c-4450-b134-7c7da19ab530 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.042102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de66b-f1bc-4d39-bfcd-9e6378653e4e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.043583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.044223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.044508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.044867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.045508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=9f85882ffc0447baaf3e00554f6b4ed1) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:24.045818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.047416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.055178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 322/656] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/role_assignments?user.id=9f85882ffc0447baaf3e00554f6b4ed1&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:24.055884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.057143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:24.091884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.092701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.093998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.094553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.111563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.113277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:24.135325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:24.136971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:24.137878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:24.137878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:24.137878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:24.137878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 335/657] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:24.138124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.203404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20659bd3-8f4e-4475-bffb-50b94a129ae4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.256000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d5048-2484-4991-8d39-f5e134026498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.257475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d5048-2484-4991-8d39-f5e134026498 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.257594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d5048-2484-4991-8d39-f5e134026498 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.257832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d5048-2484-4991-8d39-f5e134026498 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.273246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d5048-2484-4991-8d39-f5e134026498 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.275482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d5048-2484-4991-8d39-f5e134026498 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.277614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021232a2-7e01-4a43-8c68-6cb7bf54d621 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-238d5048-2484-4991-8d39-f5e134026498 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:24.295611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:24.296909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:24.297855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 323/658] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:24.298452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b3148-b163-4cef-9bab-09c9b0f29e33 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.298452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b3148-b163-4cef-9bab-09c9b0f29e33 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.298452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b3148-b163-4cef-9bab-09c9b0f29e33 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.301925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 324/659] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:24.301925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b3148-b163-4cef-9bab-09c9b0f29e33 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.337992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.338695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:24.338932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.339870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.339870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.340259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:24.340572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.342736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.359943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 336/660] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:24.359943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.391611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:24.442005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b3148-b163-4cef-9bab-09c9b0f29e33 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.476045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56556602-c71d-4e4f-acef-3e96ec552fb2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.476275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40835615-ef38-400e-868a-b8c630f84bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.476883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40835615-ef38-400e-868a-b8c630f84bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.477159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40835615-ef38-400e-868a-b8c630f84bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.477426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40835615-ef38-400e-868a-b8c630f84bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.478057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40835615-ef38-400e-868a-b8c630f84bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:24.478354 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40835615-ef38-400e-868a-b8c630f84bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.479709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40835615-ef38-400e-868a-b8c630f84bf3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.491277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 325/661] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:24.492593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e0930-a1ba-4159-95a2-f8a5b1c6fec9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.492956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e0930-a1ba-4159-95a2-f8a5b1c6fec9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.493632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e0930-a1ba-4159-95a2-f8a5b1c6fec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.513557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a59e0930-a1ba-4159-95a2-f8a5b1c6fec9 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:24.514658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a59e0930-a1ba-4159-95a2-f8a5b1c6fec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:24.515243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.516538 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.516987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.517596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.519307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:24.519784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.521393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.545355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 337/662] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:24.545355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.589589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 326/663] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:24.590454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e0930-a1ba-4159-95a2-f8a5b1c6fec9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.616317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:24.640280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:24.664821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbe69f-4dd0-441d-8472-b07cc9926c88 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.707707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.708798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.709379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.709853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.710898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=91413e6647284487bae08bb7eb02da0b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:24.711409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.713707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.728066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 338/664] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/role_assignments?user.id=91413e6647284487bae08bb7eb02da0b&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:24.728066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.743491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e0930-a1ba-4159-95a2-f8a5b1c6fec9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.803237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.804693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.805199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.805670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.814798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:24.816652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:24.837723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:24.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 10:51:24.839695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:24.848334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 327/665] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:24.853667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:24.860935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f113b0-4179-4a8a-9ba3-fb33601ef1dc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:24.910262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:24.910898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:24.911179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:24.911459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:24.933552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:24.956473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:24.988276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2569c5f-d5bd-4aca-bc34-d2a918af5d5d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.020449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.021895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.027646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.028297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.028634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.028913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.029921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:25.030233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.032205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.056314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 328/666] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:25.057161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.152363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 339/667] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:25.153843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.191883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999782b-e7c1-4297-9e89-5fcaa3f2628a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.212744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:25.232521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:25.239585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.240853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.241139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.241410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.255697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:25.262382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:25.263423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:25.264298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:25.267919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 329/668] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:25.268786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.283083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff965b-ccfc-41da-8eae-f436183028ee None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.326233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.326855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.327135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.327410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.413282 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabca0-6220-411c-81a4-5a3e7f2129f9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.418838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.420345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.462837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.463471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.463751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.464050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.475061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:25.479794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.481752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:25.502560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:25.503747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:25.504725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:25.504725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:25.504725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:25.506441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 330/669] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:25.507489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.551879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 340/670] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:25.553337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.559618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:25.622067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff4eaf-d78c-4a02-8a97-f9d38bef92e2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.658231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.659178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.659633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.660125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.660955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:25.661456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.663042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.676124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 331/671] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:25.676381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638489a-a579-4169-9499-f9323f15bb79 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.677461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.719131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.720046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.720329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.720600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.721249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:25.721554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.723476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.738780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 341/672] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:25.739998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.746426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:25.811980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334fdea-b316-4415-a2e9-7f3efd26d9a3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.827819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:25.863355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.864035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.864752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.865043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.875987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb989ea-832c-4d4b-b500-0a5047388895 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:25.880650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.885535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:25.901720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:25.902787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:25.903672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:25.903672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:25.903672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:25.906714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 332/673] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:25.907738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:25.920906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:25.922631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:25.922987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:25.923266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:25.937757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:25.939704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:25.958413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:25.959395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:25.960783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:25.960783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:25.960783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:25.960783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 342/674] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:25.960783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.022325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e7df7-5d51-4508-bc1f-b84d9d19eff0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.038030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.060096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.061123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.061600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.062214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.070014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.071824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:26.084085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:26.085009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:26.085896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:26.088380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 333/675] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:26.089348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.096795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad355f-3cd9-49ff-8cc9-33d6b05c135c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.112689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.141601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.142141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.142430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.142870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.143528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:26.143830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.146263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.163430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 343/676] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:26.165189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.221567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d155c-4945-45e1-9732-958fe0379c38 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.268266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.268898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.269178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.269455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.270163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:26.270469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.271873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.285477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 334/677] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:26.286428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.299849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3e9bd-d4bf-420f-9ae3-f840b99f7f9b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.320520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.344906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.345517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.345795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.346069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.346763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:26.347084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.349085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.367297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 344/678] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:26.369109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.377970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.406367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a55e-b3b3-4bf5-8f8d-71fedb31e4fe None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.441274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6012-819a-4f0f-89d3-08322e2e3c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.441886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6012-819a-4f0f-89d3-08322e2e3c1b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.442166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6012-819a-4f0f-89d3-08322e2e3c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.442436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6012-819a-4f0f-89d3-08322e2e3c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.449112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a6012-819a-4f0f-89d3-08322e2e3c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.450538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a6012-819a-4f0f-89d3-08322e2e3c1b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f33a6012-819a-4f0f-89d3-08322e2e3c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:26.462883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:26.463811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:26.464791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 335/679] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:26.465288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42fe4-30e4-49eb-bf92-5727d135bf4f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.465817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42fe4-30e4-49eb-bf92-5727d135bf4f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.465817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42fe4-30e4-49eb-bf92-5727d135bf4f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.467034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 336/680] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:26.472462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd42fe4-30e4-49eb-bf92-5727d135bf4f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.489149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbd95-6da3-4edb-ad0a-be4c6b2012dd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.525067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.526005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.526342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.526686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.536564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.538011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:26.549428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:26.550377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 345/681] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:26.551963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.584337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd42fe4-30e4-49eb-bf92-5727d135bf4f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.590409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.622869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7a04b-0d7d-4493-a6a0-1003c263ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.623464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7a04b-0d7d-4493-a6a0-1003c263ee0a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.623744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7a04b-0d7d-4493-a6a0-1003c263ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.624048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7a04b-0d7d-4493-a6a0-1003c263ee0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.625404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7a04b-0d7d-4493-a6a0-1003c263ee0a None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=6ecccd27dfa04cf985c9fd632ce2fe60) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:26.625712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7a04b-0d7d-4493-a6a0-1003c263ee0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.627149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7a04b-0d7d-4493-a6a0-1003c263ee0a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.642422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 337/682] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/role_assignments?user.id=6ecccd27dfa04cf985c9fd632ce2fe60&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:26.642422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388d5f3-4323-48ca-ad43-8acaf7c5153e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.642422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388d5f3-4323-48ca-ad43-8acaf7c5153e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.642422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388d5f3-4323-48ca-ad43-8acaf7c5153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.642896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.656815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e388d5f3-4323-48ca-ad43-8acaf7c5153e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:26.658050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e388d5f3-4323-48ca-ad43-8acaf7c5153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:26.674941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8bce9-6d51-4887-8557-59fb59a5c995 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.723312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.724575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.725062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.725620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.726640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:26.727159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.729120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.740828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 338/683] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:26.741869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388d5f3-4323-48ca-ad43-8acaf7c5153e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.747414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 346/684] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:26.756028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:26.917650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388d5f3-4323-48ca-ad43-8acaf7c5153e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.923861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.933088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f8c60-07bd-4805-b4f7-c90fc8b896c6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:26.961037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:26.969563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.970250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.970607 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.970951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.971935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:26.972315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.975110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:26.982932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:26.983828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:26.985109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:26.985592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:26.987512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=b5c892640b3e409b9a99b4ca1d34a964) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:26.988037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:26.989703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:27.000225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 347/685] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/role_assignments?user.id=b5c892640b3e409b9a99b4ca1d34a964&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:27.001947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 339/686] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Oct 6 10:51:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:27.002885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:27.028915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:27.143870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948c774-643e-4e0d-a64f-9172510e7b5d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:27.180986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571be53-e4b3-46d5-b7f0-c24dbddbc0f5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:27.211298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871edbf-aead-4f95-a933-257cb24fac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:27.211973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871edbf-aead-4f95-a933-257cb24fac09 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.212220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871edbf-aead-4f95-a933-257cb24fac09 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.212489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871edbf-aead-4f95-a933-257cb24fac09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.242574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b62d8-060b-4174-913f-a3eef4365428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:27.243609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b62d8-060b-4174-913f-a3eef4365428 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.244118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b62d8-060b-4174-913f-a3eef4365428 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.244585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b62d8-060b-4174-913f-a3eef4365428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.255207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:27.257297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b62d8-060b-4174-913f-a3eef4365428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:27.258809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b62d8-060b-4174-913f-a3eef4365428 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-871b62d8-060b-4174-913f-a3eef4365428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:27.275674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:27.276828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:27.277916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:27.277916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:27.277916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:27.279330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 340/687] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:27.281506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12298049-b592-419f-9812-77843b9a5acd None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.282015 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12298049-b592-419f-9812-77843b9a5acd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.282365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12298049-b592-419f-9812-77843b9a5acd None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.283992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 341/688] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:27.287957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12298049-b592-419f-9812-77843b9a5acd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:27.312109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871edbf-aead-4f95-a933-257cb24fac09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:27.313816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871edbf-aead-4f95-a933-257cb24fac09 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:27.339561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:27.412535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12298049-b592-419f-9812-77843b9a5acd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:27.451869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:27.453335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.453521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.453843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.454554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:27.454950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:27.456754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:27.458369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 348/689] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:27.460100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111e4c-3728-44d3-8a20-b297f0b59e8a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.460621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111e4c-3728-44d3-8a20-b297f0b59e8a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.461054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111e4c-3728-44d3-8a20-b297f0b59e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.479887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 342/690] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:27.479887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:27.493803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-13111e4c-3728-44d3-8a20-b297f0b59e8a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:27.495411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-13111e4c-3728-44d3-8a20-b297f0b59e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:27.563797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:27.584048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 349/691] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:27.584048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111e4c-3728-44d3-8a20-b297f0b59e8a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:27.645881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89f19d-4398-4792-bac0-213a0c355e32 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:27.647633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:27.698775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:27.699420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.699703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.700010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.724749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:27.726154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:27.727969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111e4c-3728-44d3-8a20-b297f0b59e8a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:27.756973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:27.757994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:27.758892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:27.758892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:27.758892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:27.759589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 343/692] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:27.767491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:27.787192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:27.787812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.788121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.788396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.789046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:27.790478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:27.833414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 350/693] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:51:27 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:27.932362 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7943c6-2226-443a-ba20-0a2d63593623 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:27.980719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:27.981341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:27.981621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:27.981887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:27.982536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:27.982839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:27.984293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:28.005603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 344/694] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:28.011289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.065000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.077223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:28.094526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:28.166735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d2785-f9ea-4d3f-90b3-47f94639f15d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:28.204075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fdbde-7c94-407c-b982-5da0570b978b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:28.217942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:28.218687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:28.218687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:28.219007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:28.228012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:28.229500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:28.242647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:28.243586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:28.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 351/695] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:28.250352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.251436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:28.252366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:28.252871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:28.253662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:28.268452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:28.276526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:28.277469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:28.278302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:28.281544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 345/696] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:28.287393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.411012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933ce4-bd09-496b-9710-43a870a01416 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:28.415624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:28.425076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:28.447581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a418a-5e20-4c38-9f29-a13e1edb8312 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:28.490615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:28.491327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:28.491688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:28.492085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:28.492883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:28.493205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:28.494251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:28.494727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:28.495240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:28.495710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:28.496170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:28.503862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:28.504552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:28.505450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 346/697] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:28.506503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:28.512799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.520307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 352/698] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:28.528891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.623183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5732837-a241-47f3-bafd-aa821f9dd1aa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:28.660109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:28.661205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:28.661693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:28.662164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:28.665275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:28.675288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:28.677088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:28.693614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:28.694612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:28.695505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:28.695505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:28.695505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:28.701274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 347/699] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:28.708054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.710458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550ca0d-ca17-4db6-870a-fa7a73249ef3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:28.776592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:28.777121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:28.777404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:28.777677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:28.778402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=df5aa06f9803465bb102482d351adeac) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:28.778711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:28.786597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:28.796075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 353/700] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/role_assignments?user.id=df5aa06f9803465bb102482d351adeac&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:28.877686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61dc46-beeb-40af-8a0e-5fae7cad2295 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:28.917634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:28.918262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:28.918541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:28.918821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:28.919476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:28.919785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:28.921902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:28.937025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 348/701] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:28.945044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:28.983070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:29.066664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7536a-b0ea-45a2-9194-fdb80cc223b0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:29.104876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:29.105516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.105858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.106138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.115318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:29.117103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:29.129966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:29.131011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:29.132098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 354/702] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:51:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:29.136811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:29.193678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:29.255163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f22fc7-939c-451f-9211-969b4a0b49b8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:29.292045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f792-3607-47d9-b734-d9618317dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:29.292655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f792-3607-47d9-b734-d9618317dedb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.292995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f792-3607-47d9-b734-d9618317dedb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.293266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f792-3607-47d9-b734-d9618317dedb None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.294056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f792-3607-47d9-b734-d9618317dedb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:29.294361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f792-3607-47d9-b734-d9618317dedb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:29.295801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f792-3607-47d9-b734-d9618317dedb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:29.314579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 349/703] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:51:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:29.319956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:29.324804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ee547-cb10-4196-a566-5f9f15a776f1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.325088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ee547-cb10-4196-a566-5f9f15a776f1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.325358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ee547-cb10-4196-a566-5f9f15a776f1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.326938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 350/704] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:29.333047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29e66f-39df-4c22-ab41-5a26fd9bd9b1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.333335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29e66f-39df-4c22-ab41-5a26fd9bd9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.333610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29e66f-39df-4c22-ab41-5a26fd9bd9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.355405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9d29e66f-39df-4c22-ab41-5a26fd9bd9b1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:29.356726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9d29e66f-39df-4c22-ab41-5a26fd9bd9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:29.436341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 351/705] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 10:51:29.439513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:29.449662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d246b-49c5-498d-9322-74b77d4b2e0f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:29.484391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af4fe8-3a42-4f56-9fb8-957986247b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:29.485001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af4fe8-3a42-4f56-9fb8-957986247b0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.485283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af4fe8-3a42-4f56-9fb8-957986247b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.485555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af4fe8-3a42-4f56-9fb8-957986247b0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.555292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af4fe8-3a42-4f56-9fb8-957986247b0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:29.556882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af4fe8-3a42-4f56-9fb8-957986247b0f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:29.655240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:29.671035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 355/706] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 10:51:29.680942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29e66f-39df-4c22-ab41-5a26fd9bd9b1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:29.741733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:29.780251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaac70d-b241-43f3-8b35-236578a3fdfc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.781097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaac70d-b241-43f3-8b35-236578a3fdfc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.781335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaac70d-b241-43f3-8b35-236578a3fdfc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.783236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 356/707] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:29.786344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaac70d-b241-43f3-8b35-236578a3fdfc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:29.822825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29e66f-39df-4c22-ab41-5a26fd9bd9b1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:29.858087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae208cc4-e527-4abb-a82f-62d95ff9d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:29.859437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae208cc4-e527-4abb-a82f-62d95ff9d704 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.859639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae208cc4-e527-4abb-a82f-62d95ff9d704 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.859932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae208cc4-e527-4abb-a82f-62d95ff9d704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.870629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae208cc4-e527-4abb-a82f-62d95ff9d704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:29.872107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae208cc4-e527-4abb-a82f-62d95ff9d704 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ae208cc4-e527-4abb-a82f-62d95ff9d704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:29.887207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:29.888251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:29.889155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:29.889155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:29.889155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:29.893055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 352/708] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:29.894327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcae6-9b74-40be-b008-c5450a3cebb9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.894608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcae6-9b74-40be-b008-c5450a3cebb9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.894882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcae6-9b74-40be-b008-c5450a3cebb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.913404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaac70d-b241-43f3-8b35-236578a3fdfc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:29.913725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-16abcae6-9b74-40be-b008-c5450a3cebb9 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:29.914882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-16abcae6-9b74-40be-b008-c5450a3cebb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:29.963188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:29.964258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:29.965057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:29.965432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:29.966448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:29.969532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:29.985298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:29.987080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:29.988629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:29.989629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 353/709] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:29.989629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abcae6-9b74-40be-b008-c5450a3cebb9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:29.996045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 357/710] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Oct 6 10:51:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:30.002713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.026418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:30.097517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:30.169081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abcae6-9b74-40be-b008-c5450a3cebb9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.180664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892d9a-927d-48ed-bab3-681a7e0b5d88 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.231828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.232449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.232725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.232991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.234521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:30.234819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.236207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.238845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.239469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.239716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.240010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.240640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:30.240937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.242274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.254630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 358/711] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:30.256703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 354/712] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:30.262024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.266523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.381345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:30.398260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd408-db3c-4e3d-b5a9-9d6a4d9cd7d9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.407548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebc5db-f555-41de-8e30-409a393338fa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.424859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:30.436573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.437794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.438103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.438373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.438979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:30.439277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.440762 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.446744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 359/713] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:30.447082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.447615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.448280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.448814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.449307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.458682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.469420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:30.495871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:30.496905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:30.497794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:30.498725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 355/714] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:30.501258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.610280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c08ce-1dc4-4daa-bc57-54d1e04d96e7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.647938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.648591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.648866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.649139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] PATH_INFO: `/v3/services/network` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.649788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.651272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:30.661868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:30.662947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:30.663811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 360/715] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:30.672324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.696412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c47fe-e815-4665-b316-d4991d74c5f6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.730743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:30.743144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.743751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.744060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.744335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.745157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:30.745566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.746967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.755756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:30.761567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 356/716] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:30.777506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.795104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b03f1-b8a9-4e8c-be2e-45aba106af9c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.831059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2f25e-ae1b-4cfa-8dc9-3015760ca41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.831665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2f25e-ae1b-4cfa-8dc9-3015760ca41e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.832042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2f25e-ae1b-4cfa-8dc9-3015760ca41e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.832315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2f25e-ae1b-4cfa-8dc9-3015760ca41e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.832939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2f25e-ae1b-4cfa-8dc9-3015760ca41e None admin] RBAC: Adding query filter params (name=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:30.833234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2f25e-ae1b-4cfa-8dc9-3015760ca41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.834591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2f25e-ae1b-4cfa-8dc9-3015760ca41e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.839497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 361/717] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:30.840732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792b78e-8645-4a50-a08a-7714377d6f67 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.841011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792b78e-8645-4a50-a08a-7714377d6f67 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.841283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792b78e-8645-4a50-a08a-7714377d6f67 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.842831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 362/718] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:30.848743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792b78e-8645-4a50-a08a-7714377d6f67 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:30.912375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf38d0-88c9-4592-8d62-a8ba22b4c017 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.955898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a181-bf27-480c-9dd1-de38d787974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:30.957097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a181-bf27-480c-9dd1-de38d787974e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.957672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a181-bf27-480c-9dd1-de38d787974e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.958251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a181-bf27-480c-9dd1-de38d787974e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.959089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792b78e-8645-4a50-a08a-7714377d6f67 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:30.959472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a181-bf27-480c-9dd1-de38d787974e None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=91413e6647284487bae08bb7eb02da0b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:30.960009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a181-bf27-480c-9dd1-de38d787974e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:30.962821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a181-bf27-480c-9dd1-de38d787974e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:30.970798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:30.971595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 357/719] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/role_assignments?user.id=91413e6647284487bae08bb7eb02da0b&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:30.973040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021ff43-0544-4420-b4fa-ae5762fe2b44 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:30.973552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021ff43-0544-4420-b4fa-ae5762fe2b44 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:30.974028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021ff43-0544-4420-b4fa-ae5762fe2b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:30.997001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5021ff43-0544-4420-b4fa-ae5762fe2b44 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:30.999298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5021ff43-0544-4420-b4fa-ae5762fe2b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:31.029969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:31.030658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:31.030989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:31.031247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:31.031955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] RBAC: Adding query filter params (type=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:31.032257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:31.033639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:31.045313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 363/720] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:51:30 2022] GET /identity/v3/services?type=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:31.078101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:31.091914 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 358/721] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:31.379558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:31.511620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0041-4256-47c2-8bb1-95119d65419b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:31.559995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:31.560638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:31.560919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:31.561191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:31.572542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:31.578627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:31.579663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:31.581000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 364/722] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:31.585625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021ff43-0544-4420-b4fa-ae5762fe2b44 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:31.735518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021ff43-0544-4420-b4fa-ae5762fe2b44 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:31.782080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a0d1c-9e77-4a3c-bb31-fabac2081997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:31.782693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a0d1c-9e77-4a3c-bb31-fabac2081997 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:31.782975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a0d1c-9e77-4a3c-bb31-fabac2081997 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:31.783242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a0d1c-9e77-4a3c-bb31-fabac2081997 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:31.783936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a0d1c-9e77-4a3c-bb31-fabac2081997 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:31.784238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a0d1c-9e77-4a3c-bb31-fabac2081997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:31.785626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a0d1c-9e77-4a3c-bb31-fabac2081997 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:31.800657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 359/723] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:31.808125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:31.847073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:31.947920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f4ab6-0b7a-40b8-b152-8214e6af52a8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:32.006391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:32.006971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:32.007256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:32.007536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:32.023632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:32.025123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:32.028850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82c5fa-51f3-4ba2-b8f3-2aad098f0aae None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:32.029305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82c5fa-51f3-4ba2-b8f3-2aad098f0aae None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:32.029407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82c5fa-51f3-4ba2-b8f3-2aad098f0aae None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:32.031803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 360/724] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:32.036761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0d330-891a-4add-89bd-07ffb10a5800 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:32.037049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0d330-891a-4add-89bd-07ffb10a5800 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:32.037326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0d330-891a-4add-89bd-07ffb10a5800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:32.059079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:32.060111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:32.061048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:32.061048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:32.061048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:32.066504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 365/725] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:32.070780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-eac0d330-891a-4add-89bd-07ffb10a5800 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:32.072004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eac0d330-891a-4add-89bd-07ffb10a5800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:32.074759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:32.172365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:32.225881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:32.231063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b421a-ddc2-4412-a221-28310ce1065b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:32.231922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 361/726] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:32.267759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:32.268323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:32.268608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:32.268887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:32.269764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:32.270583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:32.272142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:32.285301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 366/727] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:32.290351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0d330-891a-4add-89bd-07ffb10a5800 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:32.452066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0d330-891a-4add-89bd-07ffb10a5800 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:32.504549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:32.505183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:32.505465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:32.505738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:32.513263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:32.514793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:32.532063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:32.533021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:32.534583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 362/728] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:51:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:32.538946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:32.542144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:32.614925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:32.716911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a95a35-3c69-4d43-8ead-cdd39aea4c0f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:32.735165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6ff2-9a95-46c7-95f9-68e741819687 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:32.780256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:32.783381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:32.784738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:32.785257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:32.798451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:32.799399 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:32.799898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:32.800367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:32.801287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:32.801786 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:32.803413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:32.808044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:32.814805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:32.815733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:32.816954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:32.829912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 363/729] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:32.831381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:32.849824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 367/730] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Oct 6 10:51:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:32.857527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:32.859364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:32.979510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2b6bd-37a1-4172-b515-d0bf2e71d150 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:32.983282 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb07e7-8300-4521-90dd-b8d32d4f58eb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:32.987247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:33.023631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.024654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.025151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.025627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.026496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:33.026995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.028565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.036197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271f3a6-3311-4445-ada4-76c415a40ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.037048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271f3a6-3311-4445-ada4-76c415a40ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.037341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271f3a6-3311-4445-ada4-76c415a40ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.037615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271f3a6-3311-4445-ada4-76c415a40ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.043981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 364/731] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:33.046164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:33.164588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271f3a6-3311-4445-ada4-76c415a40ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.166483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271f3a6-3311-4445-ada4-76c415a40ad6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.174426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:33.223402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97757d5b-bbc6-4bb4-9334-03794374b3d1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:33.260118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.261091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.261596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.262081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.274634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.276600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:33.292689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:33.293653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:33.294548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:33.294548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:33.294548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:33.295432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 368/732] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:51:33.296861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3674f3-31d3-4028-a4c3-f8e16e1775e5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.297062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3674f3-31d3-4028-a4c3-f8e16e1775e5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.298118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3674f3-31d3-4028-a4c3-f8e16e1775e5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.301980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 365/734] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:33.302722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 369/734] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:33.303442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:33.303929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808ad8-7e49-4980-8afb-23bd8efc6299 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.304223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808ad8-7e49-4980-8afb-23bd8efc6299 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.304526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808ad8-7e49-4980-8afb-23bd8efc6299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.324325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:33.326097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-99808ad8-7e49-4980-8afb-23bd8efc6299 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:33.328195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-99808ad8-7e49-4980-8afb-23bd8efc6299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:33.423871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbf5a3-41f5-480c-acad-146adeb7bbd0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:33.424460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 370/735] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:33.425843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808ad8-7e49-4980-8afb-23bd8efc6299 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:33.480431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.481060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.481336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.481606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.497828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:33.505301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.506810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:33.534993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:33.536095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:33.537003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:33.537003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:33.537003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:33.537003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 366/736] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:33.544485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:33.606673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808ad8-7e49-4980-8afb-23bd8efc6299 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:33.644621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.645231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.645609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.645887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.646530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:33.646829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.648284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.661392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 371/737] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:33.667325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:33.701556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:33.742550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a624e8-6923-4fb7-b77a-ab4e32553489 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:33.768436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:33.776310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d75e0-cf6e-480a-ba84-cc909fe3acf0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:33.776945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896ff0d-2ace-4878-a0f7-543a14aa63ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.777548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896ff0d-2ace-4878-a0f7-543a14aa63ec None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.777831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896ff0d-2ace-4878-a0f7-543a14aa63ec None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.778105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896ff0d-2ace-4878-a0f7-543a14aa63ec None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.778753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896ff0d-2ace-4878-a0f7-543a14aa63ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:33.779057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896ff0d-2ace-4878-a0f7-543a14aa63ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.781487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896ff0d-2ace-4878-a0f7-543a14aa63ec None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.793176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 367/738] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:33.794353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342abbb-b72d-4aa8-b6ce-80bb8b26569f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.794684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342abbb-b72d-4aa8-b6ce-80bb8b26569f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.794910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342abbb-b72d-4aa8-b6ce-80bb8b26569f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.799880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 368/739] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:33.799880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269023c-b68a-4384-b7ad-39570111af45 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.799880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269023c-b68a-4384-b7ad-39570111af45 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.799880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269023c-b68a-4384-b7ad-39570111af45 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.799880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 369/740] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:33.799880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269023c-b68a-4384-b7ad-39570111af45 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:33.819694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.820361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.820637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.820905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.828910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.830362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:33.841500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:33.842420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:33.844111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:33.845120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 372/741] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:33.845120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:33.908905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269023c-b68a-4384-b7ad-39570111af45 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:33.937182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:33.943324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d289-2244-4453-88e4-5bc579351d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.943959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d289-2244-4453-88e4-5bc579351d4d None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.944242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d289-2244-4453-88e4-5bc579351d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.944514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d289-2244-4453-88e4-5bc579351d4d None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.945161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249d289-2244-4453-88e4-5bc579351d4d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:33.945549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fad59-f0f6-4766-a07e-5eb542e99e8f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:33.946529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249d289-2244-4453-88e4-5bc579351d4d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:33.962479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 370/742] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:51:33 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:33.963479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0d4b-30d4-4d6d-89fb-8300cebafdcf None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.963767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0d4b-30d4-4d6d-89fb-8300cebafdcf None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.964072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0d4b-30d4-4d6d-89fb-8300cebafdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:33.983711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-741c0d4b-30d4-4d6d-89fb-8300cebafdcf None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:33.985239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-741c0d4b-30d4-4d6d-89fb-8300cebafdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:33.997985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794ddc5-6d46-472d-995c-e0880fc62b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:33.999031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794ddc5-6d46-472d-995c-e0880fc62b4e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:33.999415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794ddc5-6d46-472d-995c-e0880fc62b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:33.999748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794ddc5-6d46-472d-995c-e0880fc62b4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.007666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794ddc5-6d46-472d-995c-e0880fc62b4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:34.009191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794ddc5-6d46-472d-995c-e0880fc62b4e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8794ddc5-6d46-472d-995c-e0880fc62b4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:34.021781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:34.022760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:34.023685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 373/743] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:34.025201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6d910-aee3-44e0-9632-de3fe7d6cc14 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.025487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6d910-aee3-44e0-9632-de3fe7d6cc14 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.025765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6d910-aee3-44e0-9632-de3fe7d6cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.046881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0af6d910-aee3-44e0-9632-de3fe7d6cc14 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:34.048150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0af6d910-aee3-44e0-9632-de3fe7d6cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:34.052486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:34.070440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 371/744] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:34.072786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0d4b-30d4-4d6d-89fb-8300cebafdcf None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.151181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 374/745] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 10:51:34.152768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6d910-aee3-44e0-9632-de3fe7d6cc14 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.190271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0d4b-30d4-4d6d-89fb-8300cebafdcf None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:34.217552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:34.239123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:34.240010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.240303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.240581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.242237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:34.242558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:34.244040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:34.267721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 372/746] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:34.273718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.284109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6d910-aee3-44e0-9632-de3fe7d6cc14 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:34.329298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:34.330284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.331063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.331542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.333166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:34.333683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:34.335326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:34.349570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:34.353392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 375/747] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:34.354229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.403476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f87dfa-18c7-4706-aba3-043a4008c4c3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:34.453020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:34.453954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.454427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.455383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.481450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8418e38-beaf-4162-a0b5-1186057b8865 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:34.510741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:34.520442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:34.521114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.521401 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.521722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.533547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:34.535047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:34.536537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:34.538782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:34.539668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:34.540578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 376/748] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:34.542668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.604318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:34.649462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 373/749] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 10:51:34.650295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.656895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb4c59-a2b5-4612-afda-82bae6c7df4a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:34.693686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10218f7f-651e-422d-9935-951766271b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:34.694289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10218f7f-651e-422d-9935-951766271b44 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.694565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10218f7f-651e-422d-9935-951766271b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.694833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10218f7f-651e-422d-9935-951766271b44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.695555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10218f7f-651e-422d-9935-951766271b44 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=f0819d4fee38422cb796aab2df4069e3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:34.695879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10218f7f-651e-422d-9935-951766271b44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:34.697398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10218f7f-651e-422d-9935-951766271b44 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:34.703163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 377/750] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/role_assignments?user.id=f0819d4fee38422cb796aab2df4069e3&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:34.703788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10218f7f-651e-422d-9935-951766271b44 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.762934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:34.797872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b615512-f9db-429b-9b96-88eaeb3c8a4a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:34.821558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10218f7f-651e-422d-9935-951766271b44 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:34.849296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:34.850316 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:34.852235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.852773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.853462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.855059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:34.857380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:34.867177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:34.868455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:34.868943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:34.869815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:34.870610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:34.870997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:34.872815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:34.884347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 374/751] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:51:34 2022] POST /identity/v3/services => generated 240 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:34.885663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:34.888303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 378/752] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:34.889403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.019739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448c70-2ab0-4363-a1cf-d63f5f116a06 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.070331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.070978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.071261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.071537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.084385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.085875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:35.103979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:35.105001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:35.105951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:35.105951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:35.105951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:35.106146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9c443-507d-4cd6-ba8d-ea8cebcd9e27 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.106548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 379/753] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:35.114805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.160469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:35.168708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.170044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.170552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.171525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.181418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:35.187692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.190544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:35.210598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:35.211614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:35.212560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:35.212560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:35.212560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:35.214617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 375/754] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:35.220579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.243662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b833c-9f9c-410c-af89-89d1e6a177fb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.289122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.289739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.290022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.290298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.290946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:35.291258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.293235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.308691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 380/755] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:35.314918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.353163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98e90a-c65a-46fd-8941-b609e0839bbe None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.395028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.399859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.399859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.399859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.399859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:35.399859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.399859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.414478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 376/756] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:35.422667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.452253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:35.460485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d8e32-fa65-4b19-a458-1971afe56d33 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.515010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:35.523145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.523863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.524153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.524433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.536671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.538121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:35.566337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:35.567372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:35.568373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 381/757] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:35.574600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.598130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdf71-f29e-4d36-a0c9-6c5fb68813fb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.665594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.666374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.666655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.666930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.680647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.682115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:35.704377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:35.705359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:35.706276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 377/758] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:35.716810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.731955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e70b0-daf0-45a6-9ba6-1c86e389980c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.768021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.768679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.768964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.769237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.769957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:35.770261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.771678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.790728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 382/759] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:35.796949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.812112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:35.855449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac37eb3-2fb6-4956-93c2-46eeac57a00f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:35.867975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:35.903386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:35.904018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:35.904301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:35.904828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:35.906200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:35.906643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:35.908122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:35.922842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 378/760] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:35.933055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:35.966154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52741d-bd90-4896-b9a5-ac53d4e22ccc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:36.013259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:36.014197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.016095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.016574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:36.037722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:36.040408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:36.068308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:36.069329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:36.070209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:36.070209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:36.070209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:36.073157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 383/761] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:36.079100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:36.133013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069f87b-0bdd-423c-8612-b4109c2427f6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:36.196412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:36.198216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:36.198833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.204427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.204715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:36.205456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=df5aa06f9803465bb102482d351adeac) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:36.205765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:36.207215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:36.218643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 379/762] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:35 2022] GET /identity/v3/role_assignments?user.id=df5aa06f9803465bb102482d351adeac&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:36.240394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b45ce9-11b8-4999-84fe-13688150fb1a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:36.277801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:36.278435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.278719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.278994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:36.280038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:36.280948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:36.282498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:36.296717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 384/763] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:36.302614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:36.358165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:36.469048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff731f9f-21dd-40b7-87d7-ecc2133109be None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:36.526156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7235a5-0dad-4940-8e74-e2520acabd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:36.527206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7235a5-0dad-4940-8e74-e2520acabd8b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.527714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7235a5-0dad-4940-8e74-e2520acabd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.528214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7235a5-0dad-4940-8e74-e2520acabd8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:36.539660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7235a5-0dad-4940-8e74-e2520acabd8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:36.541588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7235a5-0dad-4940-8e74-e2520acabd8b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5a7235a5-0dad-4940-8e74-e2520acabd8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:36.560602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:36.562073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:36.562976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:36.565055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 380/764] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:36.571135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:36.638013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d761-40ca-4390-8ad0-2eb26ee0b09c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.638648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d761-40ca-4390-8ad0-2eb26ee0b09c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.639156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d761-40ca-4390-8ad0-2eb26ee0b09c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:36.640936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 381/765] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:36.645692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0ab0-058e-4709-8a11-4f4ed7092023 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.646046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0ab0-058e-4709-8a11-4f4ed7092023 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.646388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0ab0-058e-4709-8a11-4f4ed7092023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:36.679715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ffae0ab0-058e-4709-8a11-4f4ed7092023 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:36.682657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ffae0ab0-058e-4709-8a11-4f4ed7092023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:36.741876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4257b67-2def-47f8-9e78-c371fb7c3e7d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:36.791983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:36.814556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:36.815471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.815989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.816448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:36.817515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:36.818178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:36.821007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:36.825775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 382/766] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:36.836268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 385/767] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:36.842341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae0ab0-058e-4709-8a11-4f4ed7092023 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:36.869069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:36.948328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae0ab0-058e-4709-8a11-4f4ed7092023 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:36.997039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197704fe-122b-4800-9566-541c5a85274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:36.997984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197704fe-122b-4800-9566-541c5a85274a None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:36.998462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197704fe-122b-4800-9566-541c5a85274a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:36.998955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197704fe-122b-4800-9566-541c5a85274a None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/91413e6647284487bae08bb7eb02da0b/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.005140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197704fe-122b-4800-9566-541c5a85274a None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=91413e6647284487bae08bb7eb02da0b, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:37.006915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197704fe-122b-4800-9566-541c5a85274a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:37.025483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 383/768] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:36 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/91413e6647284487bae08bb7eb02da0b/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 10:51:37.134207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:37.236473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4882608-12aa-49da-b98f-f3b6195063c2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:37.236990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4882608-12aa-49da-b98f-f3b6195063c2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:37.237108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4882608-12aa-49da-b98f-f3b6195063c2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.238619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 384/769] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:37.243688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c6e59-b049-4a6f-8816-0e26a72d790b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:37.244023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c6e59-b049-4a6f-8816-0e26a72d790b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:37.244304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c6e59-b049-4a6f-8816-0e26a72d790b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.271679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe8db4-bf76-40ab-a448-9e95bde36624 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:37.272951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-630c6e59-b049-4a6f-8816-0e26a72d790b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:37.274351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-630c6e59-b049-4a6f-8816-0e26a72d790b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:37.310939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:37.311763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:37.312181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:37.312462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.313055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:37.315107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:37.325032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:37.326049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:37.326895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:37.330643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 386/770] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Oct 6 10:51:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:37.337904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:37.369655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 385/771] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:37.412445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:37.490227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acb3fc-9853-435a-87b4-5c7739932999 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:37.540758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:37.541351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:37.541639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:37.541918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.542619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:37.542926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:37.544378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:37.549591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 387/772] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Oct 6 10:51:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:37.555273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c6e59-b049-4a6f-8816-0e26a72d790b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:37.571310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:37.675253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c6e59-b049-4a6f-8816-0e26a72d790b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:37.712774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:37.713386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:37.713667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:37.713939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.714627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:37.714936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:37.716335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:37.723110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 386/773] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Oct 6 10:51:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:37.728891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:37.739815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:37.871047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c99570-1871-4707-9d80-2e2ac5d8674b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:37.893385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:37.919213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03794db-7965-4036-8317-7ae800c5679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:37.920244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03794db-7965-4036-8317-7ae800c5679c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:37.920728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03794db-7965-4036-8317-7ae800c5679c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:37.921402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03794db-7965-4036-8317-7ae800c5679c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.922335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03794db-7965-4036-8317-7ae800c5679c None admin] RBAC: Adding query filter params (interface=public, service_id=61858d8f5082496db0636bbda294c2ab, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:37.922894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03794db-7965-4036-8317-7ae800c5679c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:37.924537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03794db-7965-4036-8317-7ae800c5679c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:37.939459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 388/774] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:51:37 2022] GET /identity/v3/endpoints?service_id=61858d8f5082496db0636bbda294c2ab&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:37.941190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b54c9-ffa5-41d5-96d7-f1d2072e89fa None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:37.941680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b54c9-ffa5-41d5-96d7-f1d2072e89fa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:37.942181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b54c9-ffa5-41d5-96d7-f1d2072e89fa None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:37.948052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 389/775] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:37.949266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b54c9-ffa5-41d5-96d7-f1d2072e89fa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:37.987668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:38.000943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72838471-7061-453c-bc79-834dcd8baf36 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:38.037834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbff94-21a7-4e68-b057-ab984f87a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:38.038564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbff94-21a7-4e68-b057-ab984f87a5df None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:38.038939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbff94-21a7-4e68-b057-ab984f87a5df None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:38.039302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbff94-21a7-4e68-b057-ab984f87a5df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:38.051252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbff94-21a7-4e68-b057-ab984f87a5df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-05fbff94-21a7-4e68-b057-ab984f87a5df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:38.056373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:38.057390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:38.058428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 387/776] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:38.060970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d317ec8-88a6-4485-be3e-731212ad0f45 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:38.061141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d317ec8-88a6-4485-be3e-731212ad0f45 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:38.061569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d317ec8-88a6-4485-be3e-731212ad0f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:38.080474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3d317ec8-88a6-4485-be3e-731212ad0f45 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:38.081911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3d317ec8-88a6-4485-be3e-731212ad0f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:38.082764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b54c9-ffa5-41d5-96d7-f1d2072e89fa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:38.127531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:38.128847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:38.129273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:38.130089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:38.130988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:38.132733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:38.139214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 390/777] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:51:37 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:38.140242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:38.176876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 388/778] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:38.210584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:38.287200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2831e52-a38b-4bc8-bfb2-c85372b3fdda None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:38.332532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:38.333442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:38.333856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:38.334273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:38.335085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:38.335528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:38.337079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:38.346676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 391/779] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:38.352942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d317ec8-88a6-4485-be3e-731212ad0f45 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:38.439903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:38.477843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d317ec8-88a6-4485-be3e-731212ad0f45 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:38.524143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:38.525076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:38.525525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:38.525965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:38.533057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:38.543317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:38.551863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:38.574837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:38.575863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:38.576753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:38.576753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:38.576753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:38.582880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 389/780] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:38.583756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:38.736843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec3ca-bcc9-4c22-8f33-995702acee95 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:38.739717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:38.781643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:38.782282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:38.782560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:38.782832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] PATH_INFO: `/v3/services/network` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:38.783420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:38.784809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:38.790912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239c50-4853-410c-a72c-c0c960e95b8e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:38.800419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:38.801424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:38.802296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:38.804518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 392/781] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:38 2022] GET /identity/v3/services/network => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:38.809529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:38.836213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:38.845403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179686de-a195-414b-9c66-82741facd239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:38.846601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179686de-a195-414b-9c66-82741facd239 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:38.847075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179686de-a195-414b-9c66-82741facd239 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:38.849389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179686de-a195-414b-9c66-82741facd239 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:38.849389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179686de-a195-414b-9c66-82741facd239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:38.849389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179686de-a195-414b-9c66-82741facd239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:38.850418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179686de-a195-414b-9c66-82741facd239 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:38.865850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 390/782] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:38.871903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179686de-a195-414b-9c66-82741facd239 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:38.993676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179686de-a195-414b-9c66-82741facd239 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.050794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c85264-773d-4599-8d21-e723f28b42bc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.066030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a48245-33b2-4cd2-b5fb-2ed017cd5682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.066934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48245-33b2-4cd2-b5fb-2ed017cd5682 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.069717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48245-33b2-4cd2-b5fb-2ed017cd5682 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.070141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48245-33b2-4cd2-b5fb-2ed017cd5682 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.086542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a48245-33b2-4cd2-b5fb-2ed017cd5682 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.088953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a48245-33b2-4cd2-b5fb-2ed017cd5682 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-41a48245-33b2-4cd2-b5fb-2ed017cd5682 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:39.113804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:39.115032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:39.116234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:39.125066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 391/783] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:51:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:39.126522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff07f-e35c-4a94-990c-3349c7f99741 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.128425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff07f-e35c-4a94-990c-3349c7f99741 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.128923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff07f-e35c-4a94-990c-3349c7f99741 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.129030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ba7fe-308c-4aa3-996b-b6fd419ea403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.129667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ba7fe-308c-4aa3-996b-b6fd419ea403 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.129953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ba7fe-308c-4aa3-996b-b6fd419ea403 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.130228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ba7fe-308c-4aa3-996b-b6fd419ea403 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.130943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ba7fe-308c-4aa3-996b-b6fd419ea403 None admin] RBAC: Adding query filter params (name=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:39.131219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ba7fe-308c-4aa3-996b-b6fd419ea403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.131354 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 392/784] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:39.132257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ff07f-e35c-4a94-990c-3349c7f99741 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:39.132935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ba7fe-308c-4aa3-996b-b6fd419ea403 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.141330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 393/785] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:51:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:39.142913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c93d9-7354-46d3-9488-2598422b3981 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.143373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c93d9-7354-46d3-9488-2598422b3981 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.143817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c93d9-7354-46d3-9488-2598422b3981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.164833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b33c93d9-7354-46d3-9488-2598422b3981 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:39.166341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b33c93d9-7354-46d3-9488-2598422b3981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:39.182321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:39.196059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:39.247367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ff07f-e35c-4a94-990c-3349c7f99741 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.250281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 394/786] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:39.252613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c93d9-7354-46d3-9488-2598422b3981 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:39.328576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.329367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.329728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.330065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.330903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:39.331242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.332787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.368213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c93d9-7354-46d3-9488-2598422b3981 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.372116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 393/787] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:39.378294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:39.411600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.412573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.413189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.413637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.414725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] RBAC: Adding query filter params (type=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:39.415229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.416777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.423225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 395/788] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:39.437329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:39.480949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:39.510895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:39.517108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b567a4-a2f5-4149-b0e8-b2df4300c951 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.555819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.556850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.557311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.557787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.570029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.571736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:39.589002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:39.590213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:39.591113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:39.591113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:39.591113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:39.597988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 394/789] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:39.599869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:39.723454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7e006-4651-4a2a-b73b-e1310da00d71 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.759097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7925-2da2-40a5-bac8-c165c5c56251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.759700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7925-2da2-40a5-bac8-c165c5c56251 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.760022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7925-2da2-40a5-bac8-c165c5c56251 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.760296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7925-2da2-40a5-bac8-c165c5c56251 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.760987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7925-2da2-40a5-bac8-c165c5c56251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:39.761260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7925-2da2-40a5-bac8-c165c5c56251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.762596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7925-2da2-40a5-bac8-c165c5c56251 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.768897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5334-a7ab-4b93-8875-fa9468c0fbca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.774793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 395/790] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:39.776028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730016b6-ed7c-4a60-a046-99efe98457bc None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.776309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730016b6-ed7c-4a60-a046-99efe98457bc None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.776578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730016b6-ed7c-4a60-a046-99efe98457bc None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.778041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 396/791] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:39.778752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730016b6-ed7c-4a60-a046-99efe98457bc None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:39.794481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:39.808626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.809232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.809507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.809783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.810482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] RBAC: Adding query filter params (interface=public, service_id=25e49e79069c400fa56d0f6b5e6bd545, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:39.810789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.812444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.817597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 396/792] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/endpoints?service_id=25e49e79069c400fa56d0f6b5e6bd545&interface=public®ion_id=RegionOne => generated 194 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:39.818212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:39.895822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730016b6-ed7c-4a60-a046-99efe98457bc None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.916207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:39.929798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731f6a-e854-4631-85fd-cba89d559014 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:39.932696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dd8b6-bdf2-4b89-9fa2-0d42903ac895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.933325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd8b6-bdf2-4b89-9fa2-0d42903ac895 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.933608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd8b6-bdf2-4b89-9fa2-0d42903ac895 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.933884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd8b6-bdf2-4b89-9fa2-0d42903ac895 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.944499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dd8b6-bdf2-4b89-9fa2-0d42903ac895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-370dd8b6-bdf2-4b89-9fa2-0d42903ac895 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:39.949255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:39.955822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 397/793] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:39.956754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361beea-b419-4f1b-a816-ffa0acaa0995 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.957491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361beea-b419-4f1b-a816-ffa0acaa0995 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.957491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361beea-b419-4f1b-a816-ffa0acaa0995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.972519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e361beea-b419-4f1b-a816-ffa0acaa0995 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:39.972891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:39.973527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:39.973711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e361beea-b419-4f1b-a816-ffa0acaa0995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:39.973830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:39.974069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:39.980988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:39.982424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:39.993979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:39.994895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:39.995775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 397/794] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:39.996288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.046461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:40.059860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 398/795] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:40.059860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361beea-b419-4f1b-a816-ffa0acaa0995 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.105356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248336b-83de-4592-be54-7ddf94719786 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.145265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.146027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.146439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.146777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.147455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:40.148942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:40.160018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 398/796] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:51:39 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:40.160944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.198518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:40.219644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361beea-b419-4f1b-a816-ffa0acaa0995 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.295393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.296094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.296375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.296703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.297342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:40.297646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:40.299196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:40.317233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 399/797] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:40.323260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.376915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4cb1e-2a29-48c1-adcc-744360596516 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.417493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.418182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.418305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:40.418708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.418795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.419611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:40.420244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:40.422370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:40.437823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 399/798] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:40.438775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.462716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf5735-e572-4a8f-ad79-b57814a435d4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.501771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.502406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.502689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.502967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.515321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:40.516806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:40.522666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:40.534954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:40.536000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:40.538053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:40.538053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:40.538053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:40.538053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 400/799] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:40.538053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.563074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142339f-2a85-41d7-9a1f-816fbcc63880 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.606292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.607157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.607443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.608824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.622712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:40.630610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:40.632985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 400/800] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:40.634329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.682451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26af04-13ba-4a98-9f71-cc79ed03603c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.701933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:40.725232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.726122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.726599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.727012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/f0819d4fee38422cb796aab2df4069e3/roles/7565190481584409b1756297cec264cd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.742702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=f0819d4fee38422cb796aab2df4069e3, role_id=7565190481584409b1756297cec264cd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:40.744245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:40.755169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea59bb-ecd5-4454-892a-2b596cd9d6c0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.757219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 401/801] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:40 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/f0819d4fee38422cb796aab2df4069e3/roles/7565190481584409b1756297cec264cd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:51:40.758034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.803457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:40.811937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d0ab1-4570-4e12-a6fb-4571f9540c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.812889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d0ab1-4570-4e12-a6fb-4571f9540c42 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.813408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d0ab1-4570-4e12-a6fb-4571f9540c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.813832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d0ab1-4570-4e12-a6fb-4571f9540c42 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.814640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d0ab1-4570-4e12-a6fb-4571f9540c42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:40.815143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d0ab1-4570-4e12-a6fb-4571f9540c42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:40.817369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d0ab1-4570-4e12-a6fb-4571f9540c42 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:40.833810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 401/802] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:40.836261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab3230-3e0c-42fd-90f5-fb4309352e03 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.836728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab3230-3e0c-42fd-90f5-fb4309352e03 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.837179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab3230-3e0c-42fd-90f5-fb4309352e03 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.843602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 402/803] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:40.845235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab3230-3e0c-42fd-90f5-fb4309352e03 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:40.906082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1849b-c26d-4278-9123-126bd864eed7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.947012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e8981-93f1-49bf-bdad-52b770e07ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:40.948306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e8981-93f1-49bf-bdad-52b770e07ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.948794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e8981-93f1-49bf-bdad-52b770e07ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.949300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e8981-93f1-49bf-bdad-52b770e07ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.950192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e8981-93f1-49bf-bdad-52b770e07ac4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:40.950706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e8981-93f1-49bf-bdad-52b770e07ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:40.952754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e8981-93f1-49bf-bdad-52b770e07ac4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:40.964042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 402/804] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:40.965827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb652272-708c-4117-81d7-8c1a72e75107 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:40.966376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb652272-708c-4117-81d7-8c1a72e75107 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:40.966870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb652272-708c-4117-81d7-8c1a72e75107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:40.971370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab3230-3e0c-42fd-90f5-fb4309352e03 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:40.988872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fb652272-708c-4117-81d7-8c1a72e75107 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:40.990407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fb652272-708c-4117-81d7-8c1a72e75107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:41.013861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:41.014116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.015049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.015414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.015887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.026326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.027951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:41.047781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:41.048829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:41.049798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 403/805] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:41.050391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.082722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 403/806] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:41.083518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb652272-708c-4117-81d7-8c1a72e75107 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.126666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:41.205746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb652272-708c-4117-81d7-8c1a72e75107 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.236926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a83b4-960c-4cc5-994b-b7016ad7b013 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.246370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.246978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.247258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.247530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.248277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:41.248583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.250800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.271227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 404/807] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:41.277070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.294457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:41.330320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.331018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.331302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.331574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.357172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.359132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:41.389142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:41.390739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:41.391659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:41.391659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:41.391659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:41.391932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 404/808] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:41.398966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.419422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e8db0-27f6-4341-93ce-58fff7810a1e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.484291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.485168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.485168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.485446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.496006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.497456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:41.513316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:41.514294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:41.515696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:41.515696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:41.515696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:41.515696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 405/809] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:41.520752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.563578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888518c1-4a82-4add-9176-6900786bdd30 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.566403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:41.597388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:41.605741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.606361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.606640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.606922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.607611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:41.607881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.610250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.627995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 405/810] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:41.630312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.641489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d426ee-bdb0-4ab6-ab55-343180c9846c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.679579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.680578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.681130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.681626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.682461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:41.683060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.684762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.698588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 406/811] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:41.699787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.750748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af07c9b-f1e0-4128-9fcd-db340e43f5ce None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.792486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.793140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.793418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.793691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.794780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.796552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:41.808908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:41.810006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 10:51:41.810841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:41.815359 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 406/812] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:41.816310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.817092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e999c2-ac14-4232-b127-271cf5812f18 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.846895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:41.852780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:41.859588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.861252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.862778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.863262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:41.872743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:41.878516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:41.897038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:41.898014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:51:41.898940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:41.901224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 407/813] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:41.906655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:41.946973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167f700-1b47-4313-896a-671e10b4adde None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:41.992925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:41.993963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:41.994466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:41.994955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.005474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.007556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.013957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada4f28-6e26-4edf-9dd1-18ba767832ae None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:42.022926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:42.023818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:42.024766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:42.028048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 407/814] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:42.029655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.053779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.054401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.054726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.054944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.055571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:42.055910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.057281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.062978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 408/815] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Oct 6 10:51:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:42.063740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.123686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:42.149091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:42.167123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f9b5c-3dfd-403d-b0f2-465a47e7330a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.190857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2858-feaf-47bb-9275-85c826bf9fce None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.206650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.207340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.207687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.208063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.208857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:42.209253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.210811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.234332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 408/816] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:51:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:42.234332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.242888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.244174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.244606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.245059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.246454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:42.247033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.249489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.269177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 409/817] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:42.270704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.344010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af1ecb-8678-4ad1-9f43-59c9b0e5f133 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.381248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.381986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.382366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.382760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.383596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:42.386120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.387597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.391112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8c723-eab0-4250-b263-c29bd5e9ba6c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.398772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 409/818] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Oct 6 10:51:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:42.399746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.404724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:42.413574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:42.436688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbee6d-1deb-4eec-9607-a286bb4aa646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.438148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbee6d-1deb-4eec-9607-a286bb4aa646 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.439466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbee6d-1deb-4eec-9607-a286bb4aa646 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.439929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbee6d-1deb-4eec-9607-a286bb4aa646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.512304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e4416-03be-4f5b-9261-784cee927a0e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.514819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbee6d-1deb-4eec-9607-a286bb4aa646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.516304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbee6d-1deb-4eec-9607-a286bb4aa646 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.555502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.556200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.556477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.556745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/df5aa06f9803465bb102482d351adeac/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.558650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=df5aa06f9803465bb102482d351adeac, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.560090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.583868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 410/819] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:42 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/df5aa06f9803465bb102482d351adeac/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:51:42.583868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.656162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 410/820] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:51:42.658250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd53b5-77a3-479c-8d04-5b1ee95c72e6 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.658604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd53b5-77a3-479c-8d04-5b1ee95c72e6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.659076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd53b5-77a3-479c-8d04-5b1ee95c72e6 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.665597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 411/821] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:42.667233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd53b5-77a3-479c-8d04-5b1ee95c72e6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.670079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:42.683527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:42.714027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6f4f2-9413-4e9e-94d6-f3f3573bc8eb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.756796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19da3e9-5db1-417e-9ef8-b5fe8f0f2563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.757685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da3e9-5db1-417e-9ef8-b5fe8f0f2563 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.758107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da3e9-5db1-417e-9ef8-b5fe8f0f2563 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.758547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da3e9-5db1-417e-9ef8-b5fe8f0f2563 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.759465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19da3e9-5db1-417e-9ef8-b5fe8f0f2563 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.761137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19da3e9-5db1-417e-9ef8-b5fe8f0f2563 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.791105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 411/822] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:51:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:42.792715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87b1e0-609f-4ddf-a46d-dc07464511ba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.793165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87b1e0-609f-4ddf-a46d-dc07464511ba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.793577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87b1e0-609f-4ddf-a46d-dc07464511ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.807034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd53b5-77a3-479c-8d04-5b1ee95c72e6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:42.818406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ae87b1e0-609f-4ddf-a46d-dc07464511ba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:42.820089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae87b1e0-609f-4ddf-a46d-dc07464511ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:42.845651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:42.846294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:42.846573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:42.846844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:42.859875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:42.861359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:42.878595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:42.879641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:42.880595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:42.880595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:42.880595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:42.880595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 412/823] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:42.885967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:42.926669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 412/824] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:42.980262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:43.017901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71835-2a1f-48e9-978d-3908dd8ad01e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:43.054849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cd7a3-99f2-4d92-890c-9c488bf0b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:43.055465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cd7a3-99f2-4d92-890c-9c488bf0b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.055743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cd7a3-99f2-4d92-890c-9c488bf0b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.056045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cd7a3-99f2-4d92-890c-9c488bf0b4f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.056698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cd7a3-99f2-4d92-890c-9c488bf0b4f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:43.056999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cd7a3-99f2-4d92-890c-9c488bf0b4f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:43.058402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cd7a3-99f2-4d92-890c-9c488bf0b4f0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:43.074500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 413/825] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:43.083060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87b1e0-609f-4ddf-a46d-dc07464511ba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:43.150912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:43.201669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783e5f-5fc4-4966-be96-3dbd3a5952d2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.201929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783e5f-5fc4-4966-be96-3dbd3a5952d2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.202205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783e5f-5fc4-4966-be96-3dbd3a5952d2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.203536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 414/826] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:43.204700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87b1e0-609f-4ddf-a46d-dc07464511ba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:43.210268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef16e6b-7324-488e-9a2d-f325fdbc4504 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.210408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef16e6b-7324-488e-9a2d-f325fdbc4504 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.210696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef16e6b-7324-488e-9a2d-f325fdbc4504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.232659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cef16e6b-7324-488e-9a2d-f325fdbc4504 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:43.235105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cef16e6b-7324-488e-9a2d-f325fdbc4504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:43.272886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:43.273526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.273885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.274168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.281740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:43.283201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:43.295372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:43.296581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:43.297525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 413/827] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:43.298011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:43.318935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 415/828] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:51:43.323926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef16e6b-7324-488e-9a2d-f325fdbc4504 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:43.420407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:43.466587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec630f0-32bd-430e-987f-761280dc6145 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:43.488352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:43.504513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef16e6b-7324-488e-9a2d-f325fdbc4504 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:43.515332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:43.516029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.516312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.516685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] PATH_INFO: `/v3/services/network` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.517865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:43.519307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:43.530051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:43.531120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:43.532036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 414/829] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:43 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:43.539723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:43.576303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:43.577467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.577743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.578010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.578695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:43.578991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:43.584211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:43.607544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 416/830] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:43.610329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:43.710911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b19421-addc-40b6-a54b-49a21ec10895 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:43.738541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba447b-ee1d-4cc2-9213-a4e5287183b7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:43.748734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:43.759649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:43.760439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.760822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.761200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.761933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] RBAC: Adding query filter params (name=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:43.762338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:43.764810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:43.776203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 415/831] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:51:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:43.776203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:43.785496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:43.786113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.786396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.786671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.787400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=91413e6647284487bae08bb7eb02da0b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:43.787713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:43.789669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:43.798201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 417/832] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:43 2022] GET /identity/v3/role_assignments?user.id=91413e6647284487bae08bb7eb02da0b&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:43.799230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:43.852037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:43.933814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68b79-19c4-4590-860d-6e9d6f5e6c53 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:43.980198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:43.981210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:43.981687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:43.982152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:43.995669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:44.001960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:44.002985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:51:44.003821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:44.006592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 416/833] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:51:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:44.015222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:44.033692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08043e81-a5f4-457d-ad2e-009bc3f1e1a5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:44.073734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:44.074577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:44.074999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:44.075431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:44.076248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] RBAC: Adding query filter params (type=network) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:44.076689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:44.078796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:44.085281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 418/834] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:51:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:44.093703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:44.134261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3797fc-a721-43d1-8c5b-dd6a2e39d6d0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:44.137729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:44.138535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:44.177121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:44.177736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:44.178155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:44.178490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:44.179223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:44.180293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:44.181747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:44.191570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 417/835] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:51:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:44.200051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:44.237413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aa13c-1e4f-4db7-8038-51f555490063 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:44.284409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:44.285873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:44.286164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:44.286444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:44.287218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:44.288623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:44.326492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd082c9-ad5d-49bb-ad95-2614bf6937a2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:44.335462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 419/836] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:51:44 2022] POST /identity/v3/endpoints => generated 337 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:51:44.384971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:44.385637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:44.385921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:44.386200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:44.398911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:44.405961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:44.428511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:44.429567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:44.430471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:44.430471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:44.430471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:44.430471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 418/837] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:44.435269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:44.551161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:44.563142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff51fe0-bb00-4936-8437-ac6ca9222ebe None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:44.612602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:44.613219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:44.613495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:44.613764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:44.614416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:44.614715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:44.616109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:44.633341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 420/838] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:44.639962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:44.668333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:44.761016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e6484-4d94-4f84-ba35-5e5fff9cc9ac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:44.795173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:44.796403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:44.796680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:44.796954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:44.805730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:44.807101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:44.819534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:44.820701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:44.821864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 419/839] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:51:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:44.828970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:44.953383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9337da4-bab3-40da-9d4a-a1ff56289e1f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:44.995099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b37029-1da9-42ae-b027-574eb09a7dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:44.995881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b37029-1da9-42ae-b027-574eb09a7dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:44.996169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b37029-1da9-42ae-b027-574eb09a7dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:44.996442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b37029-1da9-42ae-b027-574eb09a7dc5 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:44.997222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b37029-1da9-42ae-b027-574eb09a7dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:44.997526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b37029-1da9-42ae-b027-574eb09a7dc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:44.999131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b37029-1da9-42ae-b027-574eb09a7dc5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:45.023158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 421/840] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Oct 6 10:51:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:45.028790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:45.115460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:45.143404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f846e04-339c-4e2c-b9f7-bae0b6cbd9ff None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:45.180465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:45.181126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.181375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.182735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.214702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40a63-5f2e-47a3-8223-02e4e4e10fa1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.215321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40a63-5f2e-47a3-8223-02e4e4e10fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.215801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40a63-5f2e-47a3-8223-02e4e4e10fa1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.217748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 422/841] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:45.222973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905063e-94cb-4b79-a77c-1158024779f0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.223525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905063e-94cb-4b79-a77c-1158024779f0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.224037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905063e-94cb-4b79-a77c-1158024779f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.245598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4905063e-94cb-4b79-a77c-1158024779f0 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:45.247803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4905063e-94cb-4b79-a77c-1158024779f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:45.252147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:45.253985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:45.255514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:45.333518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:45.352538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 423/842] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:51:45.366911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 420/843] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:51:45.372923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905063e-94cb-4b79-a77c-1158024779f0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:45.488568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905063e-94cb-4b79-a77c-1158024779f0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:45.524455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af23ea-6875-4380-874d-b766ca1abd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:45.525322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af23ea-6875-4380-874d-b766ca1abd40 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.525744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af23ea-6875-4380-874d-b766ca1abd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.526205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af23ea-6875-4380-874d-b766ca1abd40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.534903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:45.536864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af23ea-6875-4380-874d-b766ca1abd40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:45.538742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af23ea-6875-4380-874d-b766ca1abd40 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e2af23ea-6875-4380-874d-b766ca1abd40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:45.554312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:45.555750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:45.556712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:45.556712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:45.556712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:45.556712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 424/844] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:45.562013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:45.583271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d819-360f-419a-8d16-93ab7ae834d3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.583565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d819-360f-419a-8d16-93ab7ae834d3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.583830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d819-360f-419a-8d16-93ab7ae834d3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.585444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 425/845] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:45.590245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62df18f-84b4-4621-a666-3da90ace92a2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.590534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62df18f-84b4-4621-a666-3da90ace92a2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.590815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62df18f-84b4-4621-a666-3da90ace92a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.610491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b62df18f-84b4-4621-a666-3da90ace92a2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:45.611691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b62df18f-84b4-4621-a666-3da90ace92a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:45.673504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486f78-b748-44ca-af25-2070696f6e57 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:45.694059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 426/846] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:51:45.695958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62df18f-84b4-4621-a666-3da90ace92a2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:45.717970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:45.718579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.718864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.719137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.719778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:45.720912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:45.722299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:45.736771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 421/847] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:45.741745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:45.777099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:45.824145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62df18f-84b4-4621-a666-3da90ace92a2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:45.849102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12587d-f74f-4484-aeea-e4a83fd18073 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:45.850246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:45.863603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:45.864470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.864898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.865437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.877566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:45.882955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:45.884113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:51:45.885155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:45.891880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 427/848] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:45.893695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:45.896364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:45.897103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:45.897479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:45.897845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:45.905831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:45.907379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:45.919942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:45.921771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:45.922881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:45.928118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 422/849] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:45.929970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.012275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38944a-53b3-4bb6-8e73-698f62bc4c81 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.031599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:46.040635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f08c48-7b25-4d7c-8c51-02b3d14ff586 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.048623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.049323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.049508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.049768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.050430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:46.050727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.052125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.066296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 428/850] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:51:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:46.066296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.077501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.078126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.078409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.078689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.079379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:46.079689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.081186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.099927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 423/851] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:46.099927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.119046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:46.177152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1597bc-d944-4661-9ec5-2544c8206822 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.204635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486903-357a-43f1-b359-a90608b3abb5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.216141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.216776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.217095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.217891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.218461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.219939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:46.230444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:46.231444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:46.232707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 429/852] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:46.234104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.247259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.247915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.248191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.248458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.259654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.261164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:46.276682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:46.277689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:46.278615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:46.278615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:46.278615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:46.278615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 424/853] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:46.278615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.288249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:46.350701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a9433-88f3-464a-845a-9d9d28dbb68c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.360717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:46.391666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa06b5b-f20b-4ce0-bca6-4e245166fe67 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.395039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.395656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.395972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.396244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.399299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=f0819d4fee38422cb796aab2df4069e3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:46.399299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.399299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.411876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 430/854] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/role_assignments?user.id=f0819d4fee38422cb796aab2df4069e3&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:46.411876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.440681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94443c04-08be-4ea5-b393-15df9750e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.441316 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94443c04-08be-4ea5-b393-15df9750e97d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.441596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94443c04-08be-4ea5-b393-15df9750e97d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.441876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94443c04-08be-4ea5-b393-15df9750e97d None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.442524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94443c04-08be-4ea5-b393-15df9750e97d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:46.442828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94443c04-08be-4ea5-b393-15df9750e97d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.444277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94443c04-08be-4ea5-b393-15df9750e97d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.449467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 425/855] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:46.450627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5632cf-2488-4285-ac1c-f9525e85b440 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.450921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5632cf-2488-4285-ac1c-f9525e85b440 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.451197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5632cf-2488-4285-ac1c-f9525e85b440 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.456190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 426/856] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:46.456190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5632cf-2488-4285-ac1c-f9525e85b440 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.532320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:46.557380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d8b59-7589-4e5f-bcb2-a87667dda9a7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.569914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5632cf-2488-4285-ac1c-f9525e85b440 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.606848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.607471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.607754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.608359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.609415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:46.609725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.609880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6fd73-3e14-454c-862d-e49b587be7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.610475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6fd73-3e14-454c-862d-e49b587be7df None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.610760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6fd73-3e14-454c-862d-e49b587be7df None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.611038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6fd73-3e14-454c-862d-e49b587be7df None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.611128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.611675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6fd73-3e14-454c-862d-e49b587be7df None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:46.612713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6fd73-3e14-454c-862d-e49b587be7df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.613672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6fd73-3e14-454c-862d-e49b587be7df None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.617970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 427/857] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:46.619144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f87e27-b4f6-41ae-a221-69e156b10694 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.619431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f87e27-b4f6-41ae-a221-69e156b10694 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.619709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f87e27-b4f6-41ae-a221-69e156b10694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.625090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 431/858] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:46.630613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.645117 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a7f87e27-b4f6-41ae-a221-69e156b10694 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:46.646354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a7f87e27-b4f6-41ae-a221-69e156b10694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:46.676937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:46.748561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 428/859] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:46.769850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b6753-1956-4af6-a882-e1d79002ef79 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.809934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.810568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.810847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.811752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.819545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.821073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:46.833694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:46.834690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:46.835637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 432/860] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:46.840467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f87e27-b4f6-41ae-a221-69e156b10694 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:46.906528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:46.945352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f87e27-b4f6-41ae-a221-69e156b10694 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:46.979596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:46.980233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:46.980514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:46.980789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:46.981517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:46.981824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:46.984401 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.003331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 429/861] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Oct 6 10:51:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:47.009356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:47.027936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:47.120874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:47.131768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a8a0c-151d-4c22-8a2a-7b050fad2f6c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:47.164886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d1f33-df3d-4022-b928-b74ddb45cb8e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:47.168025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:47.168520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:47.168801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:47.169070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:47.200397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:47.219808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:47.221598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:47.222566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:47.223187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:47.224119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:47.226356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:47.240649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:47.241642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:47.242527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:47.245217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 430/862] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:47.253956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:47.254131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:47.254228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.354159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:47.370226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 433/863] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:51:47.371403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:47.372665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea15b8-e450-4325-88a3-5a0d3f9690bf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:47.426823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:47.428013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:47.428473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:47.429179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:47.430477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:47.430985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:47.433037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.440582 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 431/864] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:47.446633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:47.495637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe7fc5-8ff3-4f46-b682-baa224093a31 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:47.513465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:47.548378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:47.549621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:47.550122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:47.550814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:47.564224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:47.566363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.567714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94f60-f3ab-4cc8-974f-b2ed108c2aa7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:47.583288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:47.584300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:47.585178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:47.585178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:47.585178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:47.587272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 434/865] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:47.593403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:47.612829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:47.613770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:47.614052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:47.614323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:47.615180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:47.615488 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:47.617628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.627869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 432/866] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:47.658423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:47.723708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe56c7-799d-4548-8393-04e2089881b4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:47.762809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:47.763544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:47.763769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:47.764075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:47.765377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:47.765743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:47.767347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.782389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 435/867] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:47.788989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:47.899593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3584e81-8c09-479f-870f-dff4a497d2ee None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:47.914545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:47.937286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73143391-7247-49f8-892f-9256a255f29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:47.937919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73143391-7247-49f8-892f-9256a255f29d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:47.938199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73143391-7247-49f8-892f-9256a255f29d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:47.938469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73143391-7247-49f8-892f-9256a255f29d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:47.945348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73143391-7247-49f8-892f-9256a255f29d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:47.946763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73143391-7247-49f8-892f-9256a255f29d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-73143391-7247-49f8-892f-9256a255f29d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:47.958196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:47.959099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:47.960022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 433/868] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:47.965382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:48.077200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd9a58-3200-435f-98d8-60f72978d027 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:48.080059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:48.113756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9cae4-a557-4b40-82af-d9cee57cacc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:48.114718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9cae4-a557-4b40-82af-d9cee57cacc1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:48.115202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9cae4-a557-4b40-82af-d9cee57cacc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:48.115692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9cae4-a557-4b40-82af-d9cee57cacc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:48.116617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9cae4-a557-4b40-82af-d9cee57cacc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:48.117117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9cae4-a557-4b40-82af-d9cee57cacc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:48.118725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9cae4-a557-4b40-82af-d9cee57cacc1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:48.131859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 436/869] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:48.143927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73143391-7247-49f8-892f-9256a255f29d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:48.240291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73143391-7247-49f8-892f-9256a255f29d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:48.278569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c99f-5fe1-4f27-8014-99fa683fbd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:48.279810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c99f-5fe1-4f27-8014-99fa683fbd92 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:48.280336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c99f-5fe1-4f27-8014-99fa683fbd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:48.280799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c99f-5fe1-4f27-8014-99fa683fbd92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:48.281704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c99f-5fe1-4f27-8014-99fa683fbd92 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=df5aa06f9803465bb102482d351adeac) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:48.282204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c99f-5fe1-4f27-8014-99fa683fbd92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:48.283818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c99f-5fe1-4f27-8014-99fa683fbd92 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:48.295862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 434/870] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:48 2022] GET /identity/v3/role_assignments?user.id=df5aa06f9803465bb102482d351adeac&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:48.852195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97829e9-410f-4811-9e43-ca77952d295d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:48.852729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97829e9-410f-4811-9e43-ca77952d295d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:48.852729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97829e9-410f-4811-9e43-ca77952d295d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:48.854412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 437/871] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:48.861834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d59dd-a1dc-4a60-9554-1867d51cd29d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:48.862139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d59dd-a1dc-4a60-9554-1867d51cd29d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:48.862400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d59dd-a1dc-4a60-9554-1867d51cd29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:48.883332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-610d59dd-a1dc-4a60-9554-1867d51cd29d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:48.884553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-610d59dd-a1dc-4a60-9554-1867d51cd29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:48.915900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:48.958733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 435/872] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:48.965623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb9476-baff-43cf-9c36-ea142bc6071d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:48.965909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb9476-baff-43cf-9c36-ea142bc6071d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:48.966184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb9476-baff-43cf-9c36-ea142bc6071d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:48.967811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 438/873] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:48.973499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daf6e1-ad16-43b8-83ed-fcfd323c9aee None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:48.973788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daf6e1-ad16-43b8-83ed-fcfd323c9aee None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:48.974060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daf6e1-ad16-43b8-83ed-fcfd323c9aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:48.993335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f9daf6e1-ad16-43b8-83ed-fcfd323c9aee None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:48.994511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f9daf6e1-ad16-43b8-83ed-fcfd323c9aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:49.065048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 436/874] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:49.237250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb9476-baff-43cf-9c36-ea142bc6071d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:49.346697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb9476-baff-43cf-9c36-ea142bc6071d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:49.369655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daf6e1-ad16-43b8-83ed-fcfd323c9aee None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:49.391972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:49.392672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:49.392962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:49.393244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:49.407444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:49.414038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:49.415036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:51:49.416006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:49.419973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 439/875] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Oct 6 10:51:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:49.422181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:49.457823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:49.509577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:49.515810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daf6e1-ad16-43b8-83ed-fcfd323c9aee None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:49.558564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ecc75-a568-49a1-be81-1b4a6c7365e5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:49.564464 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:49.565353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:49.565888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:49.566371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:49.567772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:49.569339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:49.593749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 437/876] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:51:49 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:49.607379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:49.609259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:49.609721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:49.610210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:49.611092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:49.611575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:49.613416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:49.625392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 440/877] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Oct 6 10:51:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:49.631918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:49.756655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66b1c-599d-48c1-8f6c-d5dc88a1a4d0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:49.802094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba61dd-708a-4ce7-a4a8-8306792f9192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:49.803073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba61dd-708a-4ce7-a4a8-8306792f9192 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:49.803502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba61dd-708a-4ce7-a4a8-8306792f9192 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:49.804008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba61dd-708a-4ce7-a4a8-8306792f9192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:49.817608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba61dd-708a-4ce7-a4a8-8306792f9192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:49.819424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba61dd-708a-4ce7-a4a8-8306792f9192 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-23ba61dd-708a-4ce7-a4a8-8306792f9192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:49.835863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:49.837096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:49.838228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:49.838228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:49.838228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:49.839951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 438/878] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:49.846400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:49.909039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae321b52-2521-4902-be52-407f6c01fc36 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:49.909315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae321b52-2521-4902-be52-407f6c01fc36 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:49.909596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae321b52-2521-4902-be52-407f6c01fc36 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:49.911180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 439/879] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:49.916101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ddb7e-d5d6-4cba-a58d-37555b610d76 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:49.916385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ddb7e-d5d6-4cba-a58d-37555b610d76 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:49.916663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ddb7e-d5d6-4cba-a58d-37555b610d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:49.935145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c95ddb7e-d5d6-4cba-a58d-37555b610d76 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:49.936919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c95ddb7e-d5d6-4cba-a58d-37555b610d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:49.940796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:49.952558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:49.964413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e239f-60c7-47f0-913b-44b01cb12d34 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:50.010076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:50.011025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.011568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.012074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.012992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:50.013493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:50.015923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 440/880] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:49 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:50.016149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:50.033825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 441/881] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:50.035479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ddb7e-d5d6-4cba-a58d-37555b610d76 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:50.139749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ddb7e-d5d6-4cba-a58d-37555b610d76 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:50.176685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:50.177644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.178135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.178603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.186972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:50.188701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:50.191659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:50.202592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:50.203732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:50.204738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:50.207947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 441/882] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:51:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:50.210167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:50.300615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:50.317234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbace9e-5901-47ef-a8c1-964700e21470 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:50.362984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:50.363886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.364180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.364460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.365658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:50.365984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:50.367994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:50.392148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:50.401933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 442/883] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Oct 6 10:51:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:50.402857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:50.418811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f73d-6f93-42e8-b93b-f8589ddc1657 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:50.464089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f375ac6-fe53-45da-9c87-a6e5a40ebac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:50.464976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375ac6-fe53-45da-9c87-a6e5a40ebac5 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.465433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375ac6-fe53-45da-9c87-a6e5a40ebac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.465856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375ac6-fe53-45da-9c87-a6e5a40ebac5 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.466721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f375ac6-fe53-45da-9c87-a6e5a40ebac5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:50.469559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f375ac6-fe53-45da-9c87-a6e5a40ebac5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:50.492355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 442/884] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:51:50 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:51:50.547868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e0d8-c19f-436d-a34e-a022736060d2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:50.583266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6c63-747c-43a8-80f3-ccdf1107d48f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.583995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6c63-747c-43a8-80f3-ccdf1107d48f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.584481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6c63-747c-43a8-80f3-ccdf1107d48f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.586133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 443/885] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:50.592864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f2576-155c-424d-997f-908b0fefa03b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.593245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f2576-155c-424d-997f-908b0fefa03b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.593609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f2576-155c-424d-997f-908b0fefa03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.605881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:50.607376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.608590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.609161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.614463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c56f2576-155c-424d-997f-908b0fefa03b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:50.615149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c56f2576-155c-424d-997f-908b0fefa03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:50.679479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:50.695426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:50.700798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 444/886] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:51:50.705719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:50.706203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:50.811291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 443/887] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:51:50 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:51:50.817107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f2576-155c-424d-997f-908b0fefa03b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:50.922694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f2576-155c-424d-997f-908b0fefa03b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:50.959364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:50.959581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:50.960733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:50.961169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:50.961588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:50.977203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:50.979507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:51.000331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:51.001465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:51.007306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:51.007306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:51.007306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:51.007306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 445/888] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:51.007549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:51.055473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:51.093271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:51.099481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268bb44-c4a1-4611-aebf-bd7404a0f328 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:51.134457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baed49-dd60-4d80-85f4-ef90480897f4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:51.136750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:51.138110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.138110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.138110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.138852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:51.141154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:51.152134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:51.153368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:51:51.155294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:51.157236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 444/889] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:51.165064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:51.181134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:51.181742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.182019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.182286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.182921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:51.183217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:51.184598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.198642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 446/890] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:51.206080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:51.299549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859705f-fb73-4f7e-8d16-14b83e5404cb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:51.330171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ad995-50be-468a-bd15-622be3ad79d9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:51.332766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:51.338041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:51.338739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.339101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.339632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.340379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:51.340756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:51.343764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.349561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 445/891] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:51:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:51.354782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:51.376242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:51.377561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.378072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.378580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.383632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:51.388416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:51.390809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:51.412331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:51.413495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:51.414623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:51.420899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 447/892] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:51.421910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:51.484233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce24a-a822-4005-a67b-d1b8a11b8ff4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:51.524304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:51.524914 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.525190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.525508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.526136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:51.526435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:51.527799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.538660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 446/893] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:51:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:51.559441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285bdc1-3e12-4493-a353-ba64afefe20e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:51.600005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4f2c9-b991-4fb0-b044-e78241b82ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:51.601039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4f2c9-b991-4fb0-b044-e78241b82ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.601546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4f2c9-b991-4fb0-b044-e78241b82ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.602020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4f2c9-b991-4fb0-b044-e78241b82ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.602956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4f2c9-b991-4fb0-b044-e78241b82ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:51.603472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4f2c9-b991-4fb0-b044-e78241b82ee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:51.605188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4f2c9-b991-4fb0-b044-e78241b82ee9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.620214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 448/894] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:51.629004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:51.730248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:51.744791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f7a1-4217-4ad3-994f-608ec0eee3a1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:51.781798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f478e66-5254-473a-ac61-05a023d983dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:51.782789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478e66-5254-473a-ac61-05a023d983dd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.783298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478e66-5254-473a-ac61-05a023d983dd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.783790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478e66-5254-473a-ac61-05a023d983dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.784776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f478e66-5254-473a-ac61-05a023d983dd None admin] RBAC: Adding query filter params (role.id=a1bebaf3d4f249b99aeb6ac4cfde207a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=f0819d4fee38422cb796aab2df4069e3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:51.785325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f478e66-5254-473a-ac61-05a023d983dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:51.787013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f478e66-5254-473a-ac61-05a023d983dd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:51.793319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 447/895] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:51:51 2022] GET /identity/v3/role_assignments?user.id=f0819d4fee38422cb796aab2df4069e3&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=a1bebaf3d4f249b99aeb6ac4cfde207a => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:51.977830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c77aaa-c9c3-490f-9522-42a3f3930cc0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.978487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c77aaa-c9c3-490f-9522-42a3f3930cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.979007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c77aaa-c9c3-490f-9522-42a3f3930cc0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:51.980822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 449/896] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:51.986523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6c1f-02dd-4d57-8f57-ac9ee74086b3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:51.987405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6c1f-02dd-4d57-8f57-ac9ee74086b3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:51.987915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6c1f-02dd-4d57-8f57-ac9ee74086b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:52.008118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b97b6c1f-02dd-4d57-8f57-ac9ee74086b3 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:52.009661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b97b6c1f-02dd-4d57-8f57-ac9ee74086b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:52.081642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 448/897] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:52.352675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c77aaa-c9c3-490f-9522-42a3f3930cc0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:52.382060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:52.484773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c77aaa-c9c3-490f-9522-42a3f3930cc0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:52.522069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:52.522712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:52.522995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:52.523274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:52.523944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:52.525323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:52.535882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:52.536903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:52.537751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 450/898] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:52.543228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6c1f-02dd-4d57-8f57-ac9ee74086b3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:52.629397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:52.649560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6c1f-02dd-4d57-8f57-ac9ee74086b3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:52.687257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f830-a142-4194-9028-6569f04d263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:52.687866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f830-a142-4194-9028-6569f04d263b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:52.688171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f830-a142-4194-9028-6569f04d263b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:52.688433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f830-a142-4194-9028-6569f04d263b None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:52.689091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f830-a142-4194-9028-6569f04d263b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:52.689402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f830-a142-4194-9028-6569f04d263b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:52.690873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f830-a142-4194-9028-6569f04d263b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:52.697325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 449/899] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:52.703601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:52.792577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:52.808383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32312e-aa6c-4796-b538-5cd956ebf8bd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:52.812612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd211b-ef50-4f6e-abb5-55805f83ee33 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:52.812906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd211b-ef50-4f6e-abb5-55805f83ee33 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:52.813186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd211b-ef50-4f6e-abb5-55805f83ee33 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:52.814774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 450/900] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:52.820065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a635f7e-c815-4170-acbe-c43dc68fd96c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:52.820667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a635f7e-c815-4170-acbe-c43dc68fd96c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:52.820950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a635f7e-c815-4170-acbe-c43dc68fd96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:52.844085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3a635f7e-c815-4170-acbe-c43dc68fd96c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:52.844997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:52.845277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3a635f7e-c815-4170-acbe-c43dc68fd96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:52.846015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:52.846515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:52.846980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:52.847795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:52.848333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:52.849701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:52.855104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 451/901] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:52.861411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:52.944251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 451/902] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:51:52.981153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a82016-2dba-4f73-a125-62831e9a860e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:53.017642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:53.018305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.018590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.018863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.019563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] RBAC: Adding query filter params (interface=public, service_id=0483e5bc450a40d0b0e1a8e74c596815, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:53.019894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:53.021252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:53.026790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 452/903] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:51:52 2022] GET /identity/v3/endpoints?service_id=0483e5bc450a40d0b0e1a8e74c596815&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:53.033429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a635f7e-c815-4170-acbe-c43dc68fd96c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:53.122591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:53.149014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a635f7e-c815-4170-acbe-c43dc68fd96c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:53.187208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:53.188213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.188755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.189253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.190133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:53.191692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:53.198693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 452/904] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:51:53 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 10:51:53.253991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:53.278128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:53.383549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba533a31-4fdf-41e5-af67-7e553f8d0b8d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:53.423034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:53.423681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.423989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.424260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.424859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:53.426219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:53.435474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:53.436717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:53.438074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 453/905] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Oct 6 10:51:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:53.443281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:53.553787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95219b7-8904-4052-a1f8-54611644878a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:53.591575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:53.592784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.593260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.593814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.594527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:53.594833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:53.596234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:53.601470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 453/906] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:53.607032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:53.695146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:53.722570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430a337-d3f2-4060-b4e9-9e9cdbe0408c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:53.759602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ebac5-f652-4555-bee8-381992729be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:53.760246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ebac5-f652-4555-bee8-381992729be4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.760524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ebac5-f652-4555-bee8-381992729be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.760801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ebac5-f652-4555-bee8-381992729be4 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.761426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ebac5-f652-4555-bee8-381992729be4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:53.761730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ebac5-f652-4555-bee8-381992729be4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:53.763091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ebac5-f652-4555-bee8-381992729be4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:53.768749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 454/907] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:53.774547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:53.778444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:53.828437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07581a1-4a0f-4709-b4db-b598250c7ecf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.828911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07581a1-4a0f-4709-b4db-b598250c7ecf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.829195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07581a1-4a0f-4709-b4db-b598250c7ecf None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.831231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 455/908] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:53.835761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7db4c0-1f12-4e7d-a062-5e33c1096280 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.836062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7db4c0-1f12-4e7d-a062-5e33c1096280 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.836337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7db4c0-1f12-4e7d-a062-5e33c1096280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.858925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-aa7db4c0-1f12-4e7d-a062-5e33c1096280 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:53.860402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aa7db4c0-1f12-4e7d-a062-5e33c1096280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:53.892515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d94b8-e1dd-4052-a2db-3aa7281054e9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:53.935431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3998216-7ea8-4afd-b260-5f0f3a368558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:53.936758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3998216-7ea8-4afd-b260-5f0f3a368558 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:53.937158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3998216-7ea8-4afd-b260-5f0f3a368558 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:53.937442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3998216-7ea8-4afd-b260-5f0f3a368558 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:53.938394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3998216-7ea8-4afd-b260-5f0f3a368558 None admin] RBAC: Adding query filter params (interface=public, service_id=8d7be448307341518590bb23ea60d264, region_id=RegionOne) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:53.939180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3998216-7ea8-4afd-b260-5f0f3a368558 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:53.940999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3998216-7ea8-4afd-b260-5f0f3a368558 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:53.951278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 456/910] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:51:53.955080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 454/910] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:51:53 2022] GET /identity/v3/endpoints?service_id=8d7be448307341518590bb23ea60d264&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:53.956775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7db4c0-1f12-4e7d-a062-5e33c1096280 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:54.037520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:54.067479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d6d8-c5fd-4ecd-937f-86ba3515d392 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:54.068026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d6d8-c5fd-4ecd-937f-86ba3515d392 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:54.068104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d6d8-c5fd-4ecd-937f-86ba3515d392 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:54.069541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 455/911] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:54.073123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7db4c0-1f12-4e7d-a062-5e33c1096280 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:54.075034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b345-3c23-4afc-8e3e-188579523b3b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:54.075533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b345-3c23-4afc-8e3e-188579523b3b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:54.076045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b345-3c23-4afc-8e3e-188579523b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:54.095974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c5c1b345-3c23-4afc-8e3e-188579523b3b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:54.097460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c5c1b345-3c23-4afc-8e3e-188579523b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:54.112376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:54.112991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:54.113271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:54.113545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:54.114154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:54.115510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:54.122679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 457/912] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:51:53 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 10:51:54.175278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:54.183007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 456/913] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:54.257972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:54.367636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77246fa2-6a8b-42f0-a572-c12b03e4ad4b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:54.404566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:54.405176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:54.405456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:54.405734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:54.406391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:54.407757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:54.412170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:54.427134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 458/914] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:51:54 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:54.479406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1b345-3c23-4afc-8e3e-188579523b3b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:54.601079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1b345-3c23-4afc-8e3e-188579523b3b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:54.644230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:54.644851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:54.645133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:54.645410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:54.656698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:54.661545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:54.662453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:54.663301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 457/915] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Oct 6 10:51:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:54.668634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:54.773655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e7499-f081-43ad-8a80-0f503f6ca6c5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:54.808218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:54.808825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:54.809102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:54.809372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:54.810034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:54.810333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:54.811917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:54.821471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 459/916] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Oct 6 10:51:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:54.829179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:54.889323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:54.943962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9b673-1fd6-4fd1-a460-b19773197a93 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:54.975625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:54.976291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:54.976570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:54.976842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:54.987933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:54.989377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:55.005410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:55.006432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:55.007361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:55.007361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:55.007361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:55.007361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 458/917] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:55.013961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:55.077190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:55.128838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aef171-869b-4819-b0b3-556b900cbfa4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:55.163695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:55.164331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:55.164606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:55.164872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:55.165504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:55.165800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:55.167153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:55.178896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 460/918] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:55.184930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:55.293538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92d677-d498-4af4-93ef-4bf16bdab221 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:55.301679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:55.332072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54936a4c-c626-421f-8496-bd3b309aada3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:55.333039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54936a4c-c626-421f-8496-bd3b309aada3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:55.333527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54936a4c-c626-421f-8496-bd3b309aada3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:55.334004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54936a4c-c626-421f-8496-bd3b309aada3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:55.341779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54936a4c-c626-421f-8496-bd3b309aada3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:55.343826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54936a4c-c626-421f-8496-bd3b309aada3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-54936a4c-c626-421f-8496-bd3b309aada3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:51:55.356653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:55.357645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:51:55.358750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:55.361332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 459/919] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:51:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:55.367219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:55.487334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce14042-956f-4ed0-a1b7-4bfbe37978bf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:55.505322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:55.522633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e381b-f91d-4e9d-a8fb-fdf603a78222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:55.523242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e381b-f91d-4e9d-a8fb-fdf603a78222 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:55.523520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e381b-f91d-4e9d-a8fb-fdf603a78222 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:55.523790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e381b-f91d-4e9d-a8fb-fdf603a78222 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:55.524539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e381b-f91d-4e9d-a8fb-fdf603a78222 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:55.524838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e381b-f91d-4e9d-a8fb-fdf603a78222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:55.526256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e381b-f91d-4e9d-a8fb-fdf603a78222 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:55.544146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa35b5-367e-4c78-a532-89557de51685 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:55.544824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa35b5-367e-4c78-a532-89557de51685 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:55.544956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 461/920] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Oct 6 10:51:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:55.545492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa35b5-367e-4c78-a532-89557de51685 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:55.547065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 460/921] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:51:55.552722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa35b5-367e-4c78-a532-89557de51685 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:55.556767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b614872-55fd-444a-b009-73495f5799d4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:55.557064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b614872-55fd-444a-b009-73495f5799d4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:55.557343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b614872-55fd-444a-b009-73495f5799d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:55.579006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3b614872-55fd-444a-b009-73495f5799d4 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:55.580379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3b614872-55fd-444a-b009-73495f5799d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:55.659983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 462/922] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:55.679567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa35b5-367e-4c78-a532-89557de51685 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:55.713889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:55.714508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:55.714783 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:55.715049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:55.725871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:55.727304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:55.742369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:55.743322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:55.744675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:55.744675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:55.744675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:55.744675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 461/923] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:55.749676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b614872-55fd-444a-b009-73495f5799d4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:55.860386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b614872-55fd-444a-b009-73495f5799d4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:55.868004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:55.898383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:55.899002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:55.899289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:55.899566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:55.900250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:55.900557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:55.901933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:55.915105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 463/924] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:55.921643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:55.939089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:55.942359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:56.033383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f30cfe-1aec-4856-bffd-fd99e08753c0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.050238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35c1cb-9b1c-47a1-aac6-bba5bab0c616 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.070471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:56.071294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.071680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.072533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.082300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:56.084244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:56.086129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:56.086767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.087142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.087423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.088024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:56.089363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:56.097767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:56.098689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:56.099597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:56.100046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:56.100987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:56.101879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 464/926] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:56.103593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 462/926] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:56.104854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:56.111760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:56.164231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:56.219212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd21e72-6d37-46f2-84c3-f588cd635d56 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.221193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:56.252842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e2416-654d-4dc4-82e7-22da3bb8002e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.258947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:56.259891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.260370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.260903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.261787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:56.262310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:56.264779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:56.278652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 465/927] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:56.286501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:56.296965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:56.298273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.298775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.299265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.300456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:56.300952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:56.303068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:56.310020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 463/928] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:56.316109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:56.401516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:56.408643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15252bc8-97e0-445c-a0b2-069603204d64 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.442945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24363ab2-187b-4b3a-845f-6c8e098d750c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.446983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea48f90-e1c1-4d43-8263-6d9468820912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:56.447901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea48f90-e1c1-4d43-8263-6d9468820912 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.448376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea48f90-e1c1-4d43-8263-6d9468820912 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.448843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea48f90-e1c1-4d43-8263-6d9468820912 None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/f0819d4fee38422cb796aab2df4069e3/roles/a1bebaf3d4f249b99aeb6ac4cfde207a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.451418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea48f90-e1c1-4d43-8263-6d9468820912 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=f0819d4fee38422cb796aab2df4069e3, role_id=a1bebaf3d4f249b99aeb6ac4cfde207a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:56.453212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea48f90-e1c1-4d43-8263-6d9468820912 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:56.468130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 466/929] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:51:56 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/f0819d4fee38422cb796aab2df4069e3/roles/a1bebaf3d4f249b99aeb6ac4cfde207a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 10:51:56.469999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3df69-a846-4505-82fb-be6ddbad33b3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.470346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3df69-a846-4505-82fb-be6ddbad33b3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.470686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3df69-a846-4505-82fb-be6ddbad33b3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.472126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 467/930] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:56.478309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f039f1-9588-4ba3-8167-ca05afe4fe12 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.478649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f039f1-9588-4ba3-8167-ca05afe4fe12 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.478999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f039f1-9588-4ba3-8167-ca05afe4fe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.488203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:56.489251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.489776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.490243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.491146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:56.491679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:56.493404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:56.501118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-11f039f1-9588-4ba3-8167-ca05afe4fe12 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:56.506670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11f039f1-9588-4ba3-8167-ca05afe4fe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:56.507374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 464/931] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Oct 6 10:51:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:51:56.509740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:56.523161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:56.596295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 468/932] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:56.647606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72700358-a4cd-4b3a-8217-99cc22dd1c39 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.690320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1cef-712c-4352-bc33-5227e61d303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:56.690983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1cef-712c-4352-bc33-5227e61d303f None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.691270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1cef-712c-4352-bc33-5227e61d303f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.691544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1cef-712c-4352-bc33-5227e61d303f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.692251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd1cef-712c-4352-bc33-5227e61d303f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:56.693629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd1cef-712c-4352-bc33-5227e61d303f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:56.718033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 465/933] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:51:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:51:56.867104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f039f1-9588-4ba3-8167-ca05afe4fe12 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:56.871960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bcba7-1ee9-4de6-b204-7844b27e4231 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.872254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bcba7-1ee9-4de6-b204-7844b27e4231 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.872537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bcba7-1ee9-4de6-b204-7844b27e4231 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.874823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 466/934] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:56.880526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81115452-9e3d-4d7e-8f44-702f3b97ba24 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:56.880818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81115452-9e3d-4d7e-8f44-702f3b97ba24 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:56.881435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81115452-9e3d-4d7e-8f44-702f3b97ba24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:56.902514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-81115452-9e3d-4d7e-8f44-702f3b97ba24 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:56.904205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-81115452-9e3d-4d7e-8f44-702f3b97ba24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:56.963443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:56.971368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:56.995956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f039f1-9588-4ba3-8167-ca05afe4fe12 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:56.998534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 467/935] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:51:57.033999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.034641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.034918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.035188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.035773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:57.037253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:57.046613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:57.047574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:57.048536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 469/936] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Oct 6 10:51:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:57.054060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81115452-9e3d-4d7e-8f44-702f3b97ba24 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:57.160942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81115452-9e3d-4d7e-8f44-702f3b97ba24 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:57.197741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.198355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.198631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.198953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.199576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:57.199895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:57.202286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:57.207110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 468/937] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:57.212710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:57.287759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:57.325536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df9a96-65b6-4551-a409-af1d2a77e57d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:57.348279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:57.370283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.370901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.371186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.371462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.372124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:57.372482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:57.374445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:57.381882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 470/938] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:57.387084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:57.406496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:57.414162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07c309-b613-4c1e-9479-ce7b5be13320 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:57.457375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.458025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.458581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.458864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.459457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:57.461581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:57.474951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:57.476936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:51:57.478692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:57.487927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 469/939] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:51:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:57.487927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:57.532632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c05adc-276e-4bcd-9abe-4a7e0fe3fa4d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:57.569989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.570761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.571134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.571522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.572321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:57.573844 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:57.602651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 471/940] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:51:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:51:57.609353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f11e53-8fc5-4ecc-b9db-fccefe821b3c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:57.630365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:57.660652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.661285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.661568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.661843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.662491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:57.662797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:57.664389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:57.671133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 470/941] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:51:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:57.676958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:57.755682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:57.800544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c30b-a562-49b3-87ae-b2293f6f8e6c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:57.836001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.836613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.836894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.837168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.837804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:57.838107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:57.839465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:57.845392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 472/942] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:51:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:57.851049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:57.957883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1b0d7-8f84-40d6-9802-e7d4571d40bd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:57.995516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98b2b9-a952-449b-bc95-be743ba2090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:57.996544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98b2b9-a952-449b-bc95-be743ba2090f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:57.997021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98b2b9-a952-449b-bc95-be743ba2090f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:57.997489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98b2b9-a952-449b-bc95-be743ba2090f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:57.998562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98b2b9-a952-449b-bc95-be743ba2090f None admin] RBAC: Adding query filter params (interface=public, service_id=bf3442bc0f054e89bac1411e7f3f8eed, region_id=RegionOne) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:57.999067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98b2b9-a952-449b-bc95-be743ba2090f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:58.000663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98b2b9-a952-449b-bc95-be743ba2090f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:58.006662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 471/943] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:51:57 2022] GET /identity/v3/endpoints?service_id=bf3442bc0f054e89bac1411e7f3f8eed&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:58.012298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:58.136889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df535b5-d9d3-4c75-9d56-733a6dc884e8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:58.166987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:58.177344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d70a86-d2d1-4247-b37b-c930560bd3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:58.178536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d70a86-d2d1-4247-b37b-c930560bd3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:58.179146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d70a86-d2d1-4247-b37b-c930560bd3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:58.180575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d70a86-d2d1-4247-b37b-c930560bd3f8 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:58.181593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d70a86-d2d1-4247-b37b-c930560bd3f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:58.183627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d70a86-d2d1-4247-b37b-c930560bd3f8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:58.192033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 473/944] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:51:58 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 10:51:58.799529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6c25d-9b58-4fdc-b521-325521e159b9 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:58.800082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6c25d-9b58-4fdc-b521-325521e159b9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:58.800082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6c25d-9b58-4fdc-b521-325521e159b9 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:58.801731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 472/945] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:58.807066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eb57b-2671-44a2-8744-6a61820d407c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:58.807353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eb57b-2671-44a2-8744-6a61820d407c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:58.807627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eb57b-2671-44a2-8744-6a61820d407c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:58.826804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-715eb57b-2671-44a2-8744-6a61820d407c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:58.828087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-715eb57b-2671-44a2-8744-6a61820d407c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:58.899299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 474/946] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:59.117438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1a65-916a-4864-b150-fb4c7d03b9d3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.117781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1a65-916a-4864-b150-fb4c7d03b9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.117953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1a65-916a-4864-b150-fb4c7d03b9d3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.119478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 473/947] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:51:59.125656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5edab9-ea94-4a8f-87ff-52871de2d01e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.126115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5edab9-ea94-4a8f-87ff-52871de2d01e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.126191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5edab9-ea94-4a8f-87ff-52871de2d01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.146486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4a5edab9-ea94-4a8f-87ff-52871de2d01e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:51:59.148102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4a5edab9-ea94-4a8f-87ff-52871de2d01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:51:59.181643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f1a65-916a-4864-b150-fb4c7d03b9d3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.229857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 475/948] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:51:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:51:59.235859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:59.293351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f1a65-916a-4864-b150-fb4c7d03b9d3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:59.329644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:59.330268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.330605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.330877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.341376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:51:59.346735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:59.347657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:59.348533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 474/949] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Oct 6 10:51:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:59.354149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5edab9-ea94-4a8f-87ff-52871de2d01e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.411650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:59.461679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5edab9-ea94-4a8f-87ff-52871de2d01e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:59.499038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:59.499648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.499948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.500220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.500885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:59.501182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:59.502537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:59.511411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 476/950] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Oct 6 10:51:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:59.517484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.540400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.643752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:59.648000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dcc6b-89d0-40dc-b7ef-a1299f3e7ae2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:59.663155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a42187-cac2-4164-95ee-c10686a46104 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:59.678405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:59.696733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:59.698130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.698655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.699158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.703416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:59.704082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.704438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.704782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.705512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:59.707011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:59.709918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING keystone.server.flask.common [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 06 10:51:59.712522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:59.714551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:59.730200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 10:51:59.730200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 06 10:51:59.730200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [parameters: {'id': '75d28cda01f84947a59e7d0fcb4b8a3a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 06 10:51:59.730200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:51:59.733731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:59.734920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:51:59.736122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:51:59.736122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:51:59.736122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:51:59.737773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 475/951] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 75d28cda01f84947a59e7d0fcb4b8a3a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 75d28cda01f84947a59e7d0fcb4b8a3a. Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 10:51:59.741336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 06 10:51:59.742140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Oct 06 10:51:59.743214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 10:51:59.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 10:51:59.745126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 10:51:59.745919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'id': '75d28cda01f84947a59e7d0fcb4b8a3a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:59.746802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:59.747687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 75d28cda01f84947a59e7d0fcb4b8a3a. Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.749089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 477/952] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Oct 6 10:51:59 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:59.751023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.753525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.856272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcb8e9-ca71-406f-a9a3-650c8f554528 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:59.876019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c2c63-62c5-4959-9131-bf46746695e4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:51:59.896289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:59.897184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.897635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.898078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.898892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:51:59.899386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:59.900968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:59.914972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 476/953] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:51:59.920554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:51:59.921412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:51:59.921584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.922208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:51:59.922629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:51:59.931622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:51:59.934012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:51:59.938441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:51:59.949520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:51:59.950649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:51:59.952181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:51:59.955194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 478/954] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:51:59.960550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:51:59.996214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:00.035777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033db2cb-e4b3-4639-a19f-f06c43db2ae3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:00.069895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176b3c1-bbec-45e1-b41a-749aad886bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:00.070530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176b3c1-bbec-45e1-b41a-749aad886bff None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.070812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176b3c1-bbec-45e1-b41a-749aad886bff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.071089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176b3c1-bbec-45e1-b41a-749aad886bff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.074770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd5f1-0ef6-4a4d-a987-b08758f74dc3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:00.078750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176b3c1-bbec-45e1-b41a-749aad886bff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:00.080226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176b3c1-bbec-45e1-b41a-749aad886bff None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-9176b3c1-bbec-45e1-b41a-749aad886bff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:52:00.097534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:00.098727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:00.099698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 477/955] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:51:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:00.100348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3515ea-0cbb-419b-ad7b-a6e38c7aff96 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.100548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3515ea-0cbb-419b-ad7b-a6e38c7aff96 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.100825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3515ea-0cbb-419b-ad7b-a6e38c7aff96 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.102191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 478/956] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:00.107029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3515ea-0cbb-419b-ad7b-a6e38c7aff96 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:00.117973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7fa3d-ce62-4e8f-b12c-b49f8f75a40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:00.118947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7fa3d-ce62-4e8f-b12c-b49f8f75a40c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.119366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7fa3d-ce62-4e8f-b12c-b49f8f75a40c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.119766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7fa3d-ce62-4e8f-b12c-b49f8f75a40c None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.120658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7fa3d-ce62-4e8f-b12c-b49f8f75a40c None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:00.121098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7fa3d-ce62-4e8f-b12c-b49f8f75a40c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:00.122795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7fa3d-ce62-4e8f-b12c-b49f8f75a40c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:00.145302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 479/957] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:51:59 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:00.146782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dceb0-7bbd-4fd4-b356-2a66e125ef86 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.147011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dceb0-7bbd-4fd4-b356-2a66e125ef86 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.147285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dceb0-7bbd-4fd4-b356-2a66e125ef86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.173561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4e2dceb0-7bbd-4fd4-b356-2a66e125ef86 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:00.175151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4e2dceb0-7bbd-4fd4-b356-2a66e125ef86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:00.206001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:00.245662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3515ea-0cbb-419b-ad7b-a6e38c7aff96 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:00.271868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 480/958] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:00.297304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b503d6f-6bae-4ee8-801d-a77f15acb750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:00.298021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b503d6f-6bae-4ee8-801d-a77f15acb750 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.298309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b503d6f-6bae-4ee8-801d-a77f15acb750 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.298588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b503d6f-6bae-4ee8-801d-a77f15acb750 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.299340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b503d6f-6bae-4ee8-801d-a77f15acb750 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:00.299703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b503d6f-6bae-4ee8-801d-a77f15acb750 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:00.301230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b503d6f-6bae-4ee8-801d-a77f15acb750 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:00.322672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 479/959] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Oct 6 10:52:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:00.328629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dceb0-7bbd-4fd4-b356-2a66e125ef86 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:00.352533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:00.447689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dceb0-7bbd-4fd4-b356-2a66e125ef86 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:00.483507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:00.484206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.484490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.484808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.509801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d806-f49e-4290-96d0-16bf9968247d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.510086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d806-f49e-4290-96d0-16bf9968247d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.510361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d806-f49e-4290-96d0-16bf9968247d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.515945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 480/960] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:52:00.517851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e9762-0458-4d66-b00f-02d1115638e0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.518144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e9762-0458-4d66-b00f-02d1115638e0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.518470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e9762-0458-4d66-b00f-02d1115638e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.539375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a34e9762-0458-4d66-b00f-02d1115638e0 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:00.540650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a34e9762-0458-4d66-b00f-02d1115638e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:00.561216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:00.562730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:00.585440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:00.618600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 481/961] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:00.620256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e9762-0458-4d66-b00f-02d1115638e0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:00.674429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 481/962] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:00.679741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:00.699443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:00.747504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e9762-0458-4d66-b00f-02d1115638e0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:00.786594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:00.787573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.787915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c469-7aa2-4c42-812e-a34eabf44375 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:00.788333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.788807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.789702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:00.791384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:00.801935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:00.803119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:00.804047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:00.806890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 482/963] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:52:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:00.816757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:00.816925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:00.829720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:00.830408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.830685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.830956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.843264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:00.845196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:00.863625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:00.864680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:00.865603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:00.865603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:00.865603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:00.869379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 482/964] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:00.870218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:00.932977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e9dcd-4af9-4a26-a96c-cc07ea5015a2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:00.958983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:00.972862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:00.973466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:00.973873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:00.974147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:00.974623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:00.974922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:00.976322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:00.981613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 483/965] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:52:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:00.982203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:00.986980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda59c04-56a0-4598-9d7c-6c7d5c96d76c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.028551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.030324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.030613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.031170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.031821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:01.032145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.033517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.048392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 483/966] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:01.049704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:01.087866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:01.091530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a739cfd-f660-44b1-b742-3ce265c2f992 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.126916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.127545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.127829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.128163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.128747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.130115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:01.139492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:01.140801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:01.141827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:01.144177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 484/967] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:52:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:01.145295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:01.169365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf92739-a83b-460b-9b64-fa2d06e13188 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.206818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:01.210834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.211618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.212097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.212438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.213146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:01.213455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.214822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.220509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 484/968] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:52:01 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:01.228967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:01.275259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e96c98-954f-4e75-b17f-d1a0fdc18e2e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.325925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.327458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.327872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.328288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.336005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.337769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.347360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:52:01.350593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:01.351786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:01.352943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:01.359514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc5aec-bed4-4cf5-b79e-6c64daff326c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.360465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 485/969] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:01.361803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:01.416701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.416701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.416701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.416701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.417148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:01.417148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.417148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.423007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 485/970] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:52:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:01.428385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:01.487552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c1702-1d72-422b-8b2a-c1e593314ebd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.502623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:01.523982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.524595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.524879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.525157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.525850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:01.526159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.527739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.540869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 486/971] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:01.547345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:01.556153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb6609-8cbf-4ea1-9760-f8b4ebe850e8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.594954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.595722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.596138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.596506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.597395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:01.597800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.599261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.610149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 486/972] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:52:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:01.611373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:01.637946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:01.655497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31c874-0357-46b6-b0cc-efa22227e3ca None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.692705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab441ce-803d-448a-9243-670582067f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.693776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab441ce-803d-448a-9243-670582067f91 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.694258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab441ce-803d-448a-9243-670582067f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.694727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab441ce-803d-448a-9243-670582067f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.695798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab441ce-803d-448a-9243-670582067f91 None admin] RBAC: Adding query filter params (role.id=a1bebaf3d4f249b99aeb6ac4cfde207a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=f0819d4fee38422cb796aab2df4069e3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:01.696351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab441ce-803d-448a-9243-670582067f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.698288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab441ce-803d-448a-9243-670582067f91 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.705181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 487/973] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:01 2022] GET /identity/v3/role_assignments?user.id=f0819d4fee38422cb796aab2df4069e3&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=a1bebaf3d4f249b99aeb6ac4cfde207a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:01.728404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1dfa5-4b23-45fe-8191-638f23d0986d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:01.768121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fc68a-4cac-4a4b-9bc0-43c4026daadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:01.769096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fc68a-4cac-4a4b-9bc0-43c4026daadd None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:01.769601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fc68a-4cac-4a4b-9bc0-43c4026daadd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:01.770072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fc68a-4cac-4a4b-9bc0-43c4026daadd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:01.770994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fc68a-4cac-4a4b-9bc0-43c4026daadd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:01.772719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fc68a-4cac-4a4b-9bc0-43c4026daadd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:01.798167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 487/974] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:52:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:52:02.506465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7eac-3a71-483a-8119-07c2af5ce5cf None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:02.506982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7eac-3a71-483a-8119-07c2af5ce5cf None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:02.507060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7eac-3a71-483a-8119-07c2af5ce5cf None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:02.508621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 488/975] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:02.514670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc575e-cdb1-488b-aaab-5577caa5f0f0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:02.515150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc575e-cdb1-488b-aaab-5577caa5f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:02.515336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc575e-cdb1-488b-aaab-5577caa5f0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:02.535389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-febc575e-cdb1-488b-aaab-5577caa5f0f0 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:02.536626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-febc575e-cdb1-488b-aaab-5577caa5f0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:02.615305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 488/976] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:02.886042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f7eac-3a71-483a-8119-07c2af5ce5cf None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:02.928290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:02.997658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f7eac-3a71-483a-8119-07c2af5ce5cf None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:03.035144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:03.035756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:03.036065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:03.036393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:03.037125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:03.038571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:03.041132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING keystone.server.flask.common [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 06 10:52:03.054851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.sql.core [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 10:52:03.054851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 06 10:52:03.054851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [parameters: {'user_id': '98d136ec2bdc49478a90e26ee1ea8bbe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 06 10:52:03.054851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81404) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application conn.query(q) Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result.read() Oct 06 10:52:03.072365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 10:52:03.073319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application next(self.gen) Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application yield resource Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application next(self.gen) Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise self.value Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application session.commit() Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 10:52:03.074244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.session.flush() Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application compat.raise_( Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 10:52:03.075125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 10:52:03.077234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application conn.query(q) Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result.read() Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 10:52:03.079204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [parameters: {'user_id': '98d136ec2bdc49478a90e26ee1ea8bbe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:03.084842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 06 10:52:03.085662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 06 10:52:03.086581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 06 10:52:03.086581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 06 10:52:03.086581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 10:52:03.086581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 10:52:03.086581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 06 10:52:03.086581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:03.086581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 489/977] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Oct 6 10:52:02 2022] POST /identity/v3/users => generated 160 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:03.086991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc575e-cdb1-488b-aaab-5577caa5f0f0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:03.094537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:03.192440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc575e-cdb1-488b-aaab-5577caa5f0f0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:03.227713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea85d61-27e3-4aa0-80c2-ebbcba888549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:03.228649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea85d61-27e3-4aa0-80c2-ebbcba888549 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:03.229115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea85d61-27e3-4aa0-80c2-ebbcba888549 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:03.229568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea85d61-27e3-4aa0-80c2-ebbcba888549 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:03.237419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea85d61-27e3-4aa0-80c2-ebbcba888549 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:03.239052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea85d61-27e3-4aa0-80c2-ebbcba888549 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2ea85d61-27e3-4aa0-80c2-ebbcba888549 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:52:03.251605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:03.252624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 10:52:03.253590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:03.255742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 489/978] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:52:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:03.265345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:03.370808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:03.376492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac178-ed5b-4d4a-a64d-f47896c76a5f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:03.415894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a94fcd-ba95-4932-8287-836d8c63228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:03.416770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a94fcd-ba95-4932-8287-836d8c63228a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:03.417201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a94fcd-ba95-4932-8287-836d8c63228a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:03.417621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a94fcd-ba95-4932-8287-836d8c63228a None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:03.418462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a94fcd-ba95-4932-8287-836d8c63228a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:03.418924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a94fcd-ba95-4932-8287-836d8c63228a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:03.420848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a94fcd-ba95-4932-8287-836d8c63228a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:03.465358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 490/979] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Oct 6 10:52:03 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:03.585011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5058f-28da-4980-8182-4c7556a6be75 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:03.585534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5058f-28da-4980-8182-4c7556a6be75 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:03.585534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5058f-28da-4980-8182-4c7556a6be75 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:03.587242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 490/980] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:03.593831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7953538-c406-4c9f-aedc-6732dde5d95c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:03.594330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7953538-c406-4c9f-aedc-6732dde5d95c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:03.594428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7953538-c406-4c9f-aedc-6732dde5d95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:03.615185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c7953538-c406-4c9f-aedc-6732dde5d95c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:03.616793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c7953538-c406-4c9f-aedc-6732dde5d95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:03.692175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 491/981] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:03.964782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5058f-28da-4980-8182-4c7556a6be75 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:04.015805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8b81-de37-4774-a86f-d05ec0a1469b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.016122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8b81-de37-4774-a86f-d05ec0a1469b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.016404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8b81-de37-4774-a86f-d05ec0a1469b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.018050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 492/982] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:04.023091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f201d-67ad-4641-ac2e-91db44ad9c75 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.023225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f201d-67ad-4641-ac2e-91db44ad9c75 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.023461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f201d-67ad-4641-ac2e-91db44ad9c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.038188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:04.042029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-759f201d-67ad-4641-ac2e-91db44ad9c75 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:04.043202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-759f201d-67ad-4641-ac2e-91db44ad9c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:04.077079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5058f-28da-4980-8182-4c7556a6be75 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:04.111640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:04.112007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 493/983] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:04.112274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.112550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.112841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.113331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cc4d1-6389-4f8e-8cc7-098c7e11e802 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.113493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:04.113625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cc4d1-6389-4f8e-8cc7-098c7e11e802 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.113894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cc4d1-6389-4f8e-8cc7-098c7e11e802 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.114850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:04.115301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 494/984] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:04.120612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0ea4b-98b0-465f-bcc5-47089c069c54 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.120908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0ea4b-98b0-465f-bcc5-47089c069c54 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.121186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0ea4b-98b0-465f-bcc5-47089c069c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.136378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 491/985] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:52:03 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:52:04.144147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c0f0ea4b-98b0-465f-bcc5-47089c069c54 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:04.144147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c0f0ea4b-98b0-465f-bcc5-47089c069c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:04.155598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:04.231991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 495/986] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:04.382863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:04.486650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddac-e068-42dd-b4eb-bd06a0ad52a2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:04.507995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0ea4b-98b0-465f-bcc5-47089c069c54 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:04.512999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:04.524082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:04.524729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.525011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.525286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.525877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:04.527237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:04.536647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:04.537768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:04.538617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 492/987] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Oct 6 10:52:04 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:04.543538 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:04.616962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0ea4b-98b0-465f-bcc5-47089c069c54 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:04.652059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5ac0-fadf-4c27-b1b2-dbc2d4474fa3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:04.652573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:04.652970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.653249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.653521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.654379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:04.655747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:04.664685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:04.665690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:04.666560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 496/988] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:52:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:04.672317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:04.681476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:04.692824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:04.693706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.693991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.694324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.694963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] RBAC: Adding query filter params (name=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:04.695267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:04.696663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:04.706368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 493/989] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:04.707373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:04.765045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:04.786727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792b003-c4ee-4095-a7af-daaebfc0c598 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:04.815697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca773fc-6c85-47dd-9e70-7d04d811e7ba None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:04.823347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:04.823983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.824271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.824549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.825170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:04.825482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:04.827084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:04.832127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 497/990] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:52:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:04.837915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:04.852100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:04.852716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:04.852999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:04.853279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:04.853912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:04.854220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:04.855578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:04.861563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 494/991] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:04.925535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:04.961397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a08d97-4d2d-4398-88e7-cccb26d8d5a8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:04.999713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:05.000569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:05.000875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:05.001233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:05.002092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:05.002404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:05.003827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:05.015012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 498/992] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:52:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:05.016411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:05.128771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333faa8-8e70-4482-9809-bcb0c7b878c0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:05.147290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:05.167139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aabae-9f6c-4761-9c27-bfb16ed4d65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:05.167766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aabae-9f6c-4761-9c27-bfb16ed4d65b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:05.168285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aabae-9f6c-4761-9c27-bfb16ed4d65b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:05.168741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aabae-9f6c-4761-9c27-bfb16ed4d65b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:05.169517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aabae-9f6c-4761-9c27-bfb16ed4d65b None admin] RBAC: Adding query filter params (interface=admin, service_id=bf3442bc0f054e89bac1411e7f3f8eed, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:05.170463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aabae-9f6c-4761-9c27-bfb16ed4d65b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:05.171985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aabae-9f6c-4761-9c27-bfb16ed4d65b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:05.182085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 495/993] 10.222.0.52 () {56 vars in 1249 bytes} [Thu Oct 6 10:52:05 2022] GET /identity/v3/endpoints?service_id=bf3442bc0f054e89bac1411e7f3f8eed&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:05.183055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:05.295436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad3cf9-5329-48f5-a2df-3bef60c88605 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:05.332294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce5bf6-62ce-4b09-ab36-9316fb62bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:05.332903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5bf6-62ce-4b09-ab36-9316fb62bae5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:05.333184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5bf6-62ce-4b09-ab36-9316fb62bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:05.333457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5bf6-62ce-4b09-ab36-9316fb62bae5 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:05.334065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce5bf6-62ce-4b09-ab36-9316fb62bae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:05.335517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce5bf6-62ce-4b09-ab36-9316fb62bae5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:05.336761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:05.351863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 499/994] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:52:05 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 10:52:06.445801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64708-2550-4a4f-99f4-3a2bd4756e78 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:06.446334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64708-2550-4a4f-99f4-3a2bd4756e78 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:06.446334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64708-2550-4a4f-99f4-3a2bd4756e78 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:06.447737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 496/995] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:06.453419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10093f-a1cc-48f5-a660-3aafd62c1de6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:06.453862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10093f-a1cc-48f5-a660-3aafd62c1de6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:06.454041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10093f-a1cc-48f5-a660-3aafd62c1de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:06.472268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-db10093f-a1cc-48f5-a660-3aafd62c1de6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:06.473558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-db10093f-a1cc-48f5-a660-3aafd62c1de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:06.541841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 500/996] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:06.813653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64708-2550-4a4f-99f4-3a2bd4756e78 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:06.916127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64708-2550-4a4f-99f4-3a2bd4756e78 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:06.950739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c01c4-6c7e-4347-9e4e-3b548a182387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:06.951439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c01c4-6c7e-4347-9e4e-3b548a182387 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:06.951719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c01c4-6c7e-4347-9e4e-3b548a182387 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:06.952112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c01c4-6c7e-4347-9e4e-3b548a182387 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:06.952720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c01c4-6c7e-4347-9e4e-3b548a182387 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:06.954140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c01c4-6c7e-4347-9e4e-3b548a182387 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a79c01c4-6c7e-4347-9e4e-3b548a182387 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:06.964037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:06.965135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:06.965998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:06.967708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 497/997] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:52:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:06.971705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10093f-a1cc-48f5-a660-3aafd62c1de6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:07.077665 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10093f-a1cc-48f5-a660-3aafd62c1de6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:07.078605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc598700-6c61-4840-a739-0e411d982a71 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.079367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc598700-6c61-4840-a739-0e411d982a71 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.080815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc598700-6c61-4840-a739-0e411d982a71 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.082643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 498/998] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:07.087548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281c814-e47f-4151-8e10-46fbd4dbf31c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.087870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281c814-e47f-4151-8e10-46fbd4dbf31c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.088139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281c814-e47f-4151-8e10-46fbd4dbf31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.105802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:07.109050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d281c814-e47f-4151-8e10-46fbd4dbf31c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:07.110343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d281c814-e47f-4151-8e10-46fbd4dbf31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:07.116160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:07.117010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.117468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.117930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.119120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:07.119601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:07.121830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:07.128070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 501/999] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:52:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:07.133418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:07.161383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:07.187679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 499/1000] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:07.237633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cd822-e1da-4308-aecb-dcbfa2b42ce3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:07.271488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:07.272113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.272392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.272663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.273279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:07.273579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:07.274944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:07.284776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 502/1001] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:52:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:07.288920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281c814-e47f-4151-8e10-46fbd4dbf31c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:07.391178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281c814-e47f-4151-8e10-46fbd4dbf31c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:07.427578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:07.428228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.428512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.428781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.429482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] RBAC: Adding query filter params (interface=public, service_id=51c7c71e558b4c13a6d9f5482544e5dd, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:07.429787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:07.431138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:07.437156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 500/1002] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:52:07 2022] GET /identity/v3/endpoints?service_id=51c7c71e558b4c13a6d9f5482544e5dd&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:07.442548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:07.465035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:07.506227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:07.515388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:07.557937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22f189-4280-449b-b84e-fb55cc5f427c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:07.578010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8abfb-b725-4a00-9db2-fb54cd3b9891 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:07.594013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efd625-4dde-482a-a878-2b2b73ba8685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:07.594626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd625-4dde-482a-a878-2b2b73ba8685 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.595041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd625-4dde-482a-a878-2b2b73ba8685 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.595264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd625-4dde-482a-a878-2b2b73ba8685 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.595906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efd625-4dde-482a-a878-2b2b73ba8685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:07.597360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efd625-4dde-482a-a878-2b2b73ba8685 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:07.604400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 503/1003] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:52:07 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 10:52:07.605705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f414e-e16f-4aa0-929d-305075d7ed93 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.605992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f414e-e16f-4aa0-929d-305075d7ed93 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.606268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f414e-e16f-4aa0-929d-305075d7ed93 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.607728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 504/1004] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:07.614176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e076-252d-4a13-a037-4da1f6e4b0f7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.614462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e076-252d-4a13-a037-4da1f6e4b0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.614860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e076-252d-4a13-a037-4da1f6e4b0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.618169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:07.618798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:07.619083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:07.619358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:07.620067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:07.621478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:07.635253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-92e2e076-252d-4a13-a037-4da1f6e4b0f7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:07.636505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-92e2e076-252d-4a13-a037-4da1f6e4b0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:07.640218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 501/1005] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:52:07 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:52:07.730602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 505/1006] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:07 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:07.994772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:08.094505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:08.098860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cb8b-310d-4712-a3b4-c5f1e9b21082 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:08.134814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:08.135459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:08.135742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:08.136047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:08.136637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:08.138002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:08.147720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:08.148812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:08.149679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 502/1007] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Oct 6 10:52:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:08.155301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2e076-252d-4a13-a037-4da1f6e4b0f7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:08.203790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:08.261781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2e076-252d-4a13-a037-4da1f6e4b0f7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:08.297927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:08.298537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:08.298818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:08.299094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:08.299721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:08.300055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:08.301495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:08.312180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 506/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:52:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:08.313236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:08.424499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcc2a-7661-48bd-b9be-0efb848dea25 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:08.459779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf23ab-3a26-452b-b198-bc48f15091bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:08.460454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf23ab-3a26-452b-b198-bc48f15091bb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:08.460734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf23ab-3a26-452b-b198-bc48f15091bb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:08.461056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf23ab-3a26-452b-b198-bc48f15091bb None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:08.461690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf23ab-3a26-452b-b198-bc48f15091bb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:08.461994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf23ab-3a26-452b-b198-bc48f15091bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:08.463414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf23ab-3a26-452b-b198-bc48f15091bb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:08.473341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 503/1009] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Oct 6 10:52:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:08.474278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:08.582342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99eb22-5e94-4bfe-9522-678d7858dcd6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:08.593133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:08.618917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111ab1-8b1a-4390-983e-9068e41e95a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:08.619511 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111ab1-8b1a-4390-983e-9068e41e95a5 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:08.619788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111ab1-8b1a-4390-983e-9068e41e95a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:08.620092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111ab1-8b1a-4390-983e-9068e41e95a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:08.620786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f111ab1-8b1a-4390-983e-9068e41e95a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:08.622147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f111ab1-8b1a-4390-983e-9068e41e95a5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:08.643895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 507/1010] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:52:08 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:52:09.813646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081f44-d76b-4b6c-a7ba-84aa7bcf621b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:09.814205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081f44-d76b-4b6c-a7ba-84aa7bcf621b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:09.814205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081f44-d76b-4b6c-a7ba-84aa7bcf621b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:09.815895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 504/1011] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:09.820994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d3039-e033-4942-9c42-2d3e1c8be2c6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:09.821402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d3039-e033-4942-9c42-2d3e1c8be2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:09.821589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d3039-e033-4942-9c42-2d3e1c8be2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:09.832129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687854dc-0a06-41ef-9ac6-63a335cda51c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:09.832414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687854dc-0a06-41ef-9ac6-63a335cda51c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:09.832690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687854dc-0a06-41ef-9ac6-63a335cda51c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:09.834129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 505/1012] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:09.838939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e958fc-c8cd-408a-979a-736dd527a73f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:09.839221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e958fc-c8cd-408a-979a-736dd527a73f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:09.839492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e958fc-c8cd-408a-979a-736dd527a73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:09.840819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c74d3039-e033-4942-9c42-2d3e1c8be2c6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:09.842804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c74d3039-e033-4942-9c42-2d3e1c8be2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:09.857965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-11e958fc-c8cd-408a-979a-736dd527a73f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:09.859136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11e958fc-c8cd-408a-979a-736dd527a73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:09.882616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:09.922613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 508/1013] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:09 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:09.932186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 506/1014] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:09 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:10.189355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d3039-e033-4942-9c42-2d3e1c8be2c6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.225091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e958fc-c8cd-408a-979a-736dd527a73f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.301141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:10.307641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d3039-e033-4942-9c42-2d3e1c8be2c6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.335425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e958fc-c8cd-408a-979a-736dd527a73f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.346114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216239c-80c4-435d-884f-6201166e33df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.346754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216239c-80c4-435d-884f-6201166e33df None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.347035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216239c-80c4-435d-884f-6201166e33df None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.347309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216239c-80c4-435d-884f-6201166e33df None admin] PATH_INFO: `/v3/services/compute` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.347918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216239c-80c4-435d-884f-6201166e33df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.349292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216239c-80c4-435d-884f-6201166e33df None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3216239c-80c4-435d-884f-6201166e33df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:10.358749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:10.359777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:10.360935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 509/1015] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:10.365631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216239c-80c4-435d-884f-6201166e33df None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.376679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.377318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.377597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.377869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.378454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.379810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:10.389884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:10.390916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:10.392227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 507/1016] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:10.396685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.426075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:10.478407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216239c-80c4-435d-884f-6201166e33df None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.509159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2c8c1-2944-4f82-82c4-bf82104d77d5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.515567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.516201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.516481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.516753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.517376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:10.517674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.519032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.524145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 510/1017] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:10.529883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.549337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.549954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.550235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.550512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.551138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] RBAC: Adding query filter params (name=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:10.551441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.552854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.556952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:10.564675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 508/1018] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:10.565883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.636069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecfb5e-b6f4-41f3-bd9b-8a47530a6646 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.663701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:10.673497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.674370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.674883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.675352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.676176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:10.676680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.678323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.680487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63088b-9b15-4297-8263-e4ae0c578b02 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.688733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 511/1019] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:10.690805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.716397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.716936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.717333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.717869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.718665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:10.718895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.720345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.726109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 509/1020] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:10.732669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.795427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:10.809252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b94d0-427a-4472-b405-827193e90e7a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.841256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f3c7f-6a69-41f3-813c-8ca2fbd2dcf6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:10.846273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e816800-bd7e-4e64-99f5-16996ef0ca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.846877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816800-bd7e-4e64-99f5-16996ef0ca4d None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.847154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816800-bd7e-4e64-99f5-16996ef0ca4d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.847422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816800-bd7e-4e64-99f5-16996ef0ca4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.848112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e816800-bd7e-4e64-99f5-16996ef0ca4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.849473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e816800-bd7e-4e64-99f5-16996ef0ca4d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.869483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 512/1021] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:52:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:52:10.870703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaaf71-01e4-4085-8110-31775a19752f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.870993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaaf71-01e4-4085-8110-31775a19752f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.871267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaaf71-01e4-4085-8110-31775a19752f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.877550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 513/1022] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:10.879725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74260750-ab2d-4e48-a024-056d9ee36b47 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.880053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74260750-ab2d-4e48-a024-056d9ee36b47 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.880344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74260750-ab2d-4e48-a024-056d9ee36b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.885849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:10.886570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:10.886919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:10.887261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:10.888514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] RBAC: Adding query filter params (interface=public, service_id=c75c6a555a4f4b24be131f223f90b8a4, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:10.888977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:10.891395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:10.902984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 510/1023] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/endpoints?service_id=c75c6a555a4f4b24be131f223f90b8a4&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:10.904961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-74260750-ab2d-4e48-a024-056d9ee36b47 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:10.905118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:10.906389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-74260750-ab2d-4e48-a024-056d9ee36b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:10.921822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:10.996220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 514/1024] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:11.021304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce52376-0fa5-4b31-8dd3-eceae1c3435a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:11.058491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:11.059106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:11.059379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:11.059652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:11.060517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:11.062031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:11.069742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 511/1025] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:52:10 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 10:52:11.265164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74260750-ab2d-4e48-a024-056d9ee36b47 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:11.343010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:11.370374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74260750-ab2d-4e48-a024-056d9ee36b47 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:11.411528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031b8e0-2a0f-491b-8dfd-f9cd21f557b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:11.412202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031b8e0-2a0f-491b-8dfd-f9cd21f557b8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:11.412504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031b8e0-2a0f-491b-8dfd-f9cd21f557b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:11.412776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031b8e0-2a0f-491b-8dfd-f9cd21f557b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:11.413866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031b8e0-2a0f-491b-8dfd-f9cd21f557b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:11.415295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031b8e0-2a0f-491b-8dfd-f9cd21f557b8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:11.416887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 515/1026] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:11.422856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:11.500974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:11.527428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad552931-19ff-44fd-9e1e-1e138d36c052 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:11.563599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2d4e-0d93-42a9-8492-91bc1df5cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:11.563599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2d4e-0d93-42a9-8492-91bc1df5cf24 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:11.563599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2d4e-0d93-42a9-8492-91bc1df5cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:11.563599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2d4e-0d93-42a9-8492-91bc1df5cf24 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:11.564123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2d4e-0d93-42a9-8492-91bc1df5cf24 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:11.565563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2d4e-0d93-42a9-8492-91bc1df5cf24 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:11.603865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 512/1027] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:52:11 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:52:13.208687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110145cb-f0ed-44dd-a137-a0b475805cf3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.209232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110145cb-f0ed-44dd-a137-a0b475805cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.209232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110145cb-f0ed-44dd-a137-a0b475805cf3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.210723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 516/1028] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:13.215756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd8f7-882e-4251-857e-54e941b54f8f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.216191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd8f7-882e-4251-857e-54e941b54f8f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.216469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd8f7-882e-4251-857e-54e941b54f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.235097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c95bd8f7-882e-4251-857e-54e941b54f8f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:13.236341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c95bd8f7-882e-4251-857e-54e941b54f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:13.264317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013392f-37b5-4d1d-9e16-42b9f552d96f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.264615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013392f-37b5-4d1d-9e16-42b9f552d96f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.264869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013392f-37b5-4d1d-9e16-42b9f552d96f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.266352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 517/1029] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:52:13.271211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b917d4-2fe3-46b3-8a6e-e231efff55eb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.271499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b917d4-2fe3-46b3-8a6e-e231efff55eb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.271780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b917d4-2fe3-46b3-8a6e-e231efff55eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.290152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c5b917d4-2fe3-46b3-8a6e-e231efff55eb None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:13.291344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c5b917d4-2fe3-46b3-8a6e-e231efff55eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:13.305914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 513/1030] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:13.359532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 518/1031] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:13.581119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd8f7-882e-4251-857e-54e941b54f8f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:13.629279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b917d4-2fe3-46b3-8a6e-e231efff55eb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:13.641283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:13.693187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd8f7-882e-4251-857e-54e941b54f8f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:13.705727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:13.733199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:13.733919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.734183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.734456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.738740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b917d4-2fe3-46b3-8a6e-e231efff55eb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:13.745303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:13.751225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:13.752216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:13.753089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 514/1032] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Oct 6 10:52:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:13.758227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:13.780825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:13.781463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.781746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.782124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.782709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:13.784107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:13.792819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:13.793821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:13.795549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 519/1033] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Oct 6 10:52:13 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:13.800466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:13.864946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b11b7a-37ce-4c41-82e2-e6eafc4794de None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:13.880844 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:13.898581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dd290-564f-48e5-96fd-646025be0032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:13.899187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd290-564f-48e5-96fd-646025be0032 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.899466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd290-564f-48e5-96fd-646025be0032 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.899738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd290-564f-48e5-96fd-646025be0032 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.900432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dd290-564f-48e5-96fd-646025be0032 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:13.900649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f86177-cc3d-4f45-aea7-4af6f90becb5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:13.900833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dd290-564f-48e5-96fd-646025be0032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:13.902081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dd290-564f-48e5-96fd-646025be0032 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:13.910901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 515/1034] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Oct 6 10:52:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:13.912099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37957e11-b148-4e77-b5be-0dd2ac6e627c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.912385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37957e11-b148-4e77-b5be-0dd2ac6e627c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.912658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37957e11-b148-4e77-b5be-0dd2ac6e627c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.913839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 516/1035] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:13.918661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37957e11-b148-4e77-b5be-0dd2ac6e627c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:13.939381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7fecf-3d18-4d86-b4f9-4d8868a8e16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:13.940014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7fecf-3d18-4d86-b4f9-4d8868a8e16d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.940298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7fecf-3d18-4d86-b4f9-4d8868a8e16d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.940569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7fecf-3d18-4d86-b4f9-4d8868a8e16d None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.941196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7fecf-3d18-4d86-b4f9-4d8868a8e16d None admin] RBAC: Adding query filter params (name=image) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:13.941496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7fecf-3d18-4d86-b4f9-4d8868a8e16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:13.942858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7fecf-3d18-4d86-b4f9-4d8868a8e16d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:13.944655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:13.948655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 520/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:13.949853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17441ac9-6cff-480d-a1c0-4468b1bc1ffd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:13.950137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17441ac9-6cff-480d-a1c0-4468b1bc1ffd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:13.950410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17441ac9-6cff-480d-a1c0-4468b1bc1ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:13.968451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-17441ac9-6cff-480d-a1c0-4468b1bc1ffd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:13.969627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-17441ac9-6cff-480d-a1c0-4468b1bc1ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:14.025104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37957e11-b148-4e77-b5be-0dd2ac6e627c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.035385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 521/1037] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:14.036251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17441ac9-6cff-480d-a1c0-4468b1bc1ffd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.060569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.061207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.061485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.061757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.072035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.073472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:14.088222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:14.089191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:14.090102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:14.090102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:14.090102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:14.090102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 517/1038] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:14.095141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.127981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:14.148710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17441ac9-6cff-480d-a1c0-4468b1bc1ffd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.178482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:14.186536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.187172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.187456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.187732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.188391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:14.188697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.190092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.199698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 522/1039] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:14.201440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.210128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9be9-2908-4329-9f9b-ed4d1e0b1e54 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.250382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.251259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.251556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.251863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.252903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:14.253414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.255074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.268458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 518/1040] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:14.273858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.311450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebfddb-8907-4543-b290-107f189827a7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.347356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.347990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.348279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.348557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.349218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.350986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.366220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:14.376638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 523/1041] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:52:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:52:14.376638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.382820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738e1eb-f1b1-45a4-85f0-f668b986197e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.436470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.437127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.437410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.437741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.445812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.447282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.450265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:52:14.461847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:14.463105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:52:14.464333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:14.466394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 519/1042] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:52:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:14.472841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.515358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede6d81-c1cc-43bd-877f-ae0bec2c217e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.553176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.553807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.554087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.554358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.555490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.556961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.558847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 524/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:14.565029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.590587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5127d-59eb-4c91-bfd5-da6dc585d4b6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.630533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.631143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.631423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.631695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.632484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:14.632768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.634249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.652082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 520/1044] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:52:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:14.660696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:14.664885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:14.691590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce13aa-8667-4755-9002-4a105606f7d4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.709241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:14.729122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.729725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.730009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.730283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.730932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.733288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.760069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 525/1045] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:52:14 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:52:14.772951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad2007-9cd9-40f6-97bd-689b8c1fba17 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:14.813070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:14.813710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:14.813995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:14.814287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:14.888474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:14.889867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:14.893776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:15.000501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 521/1046] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 10:52:15.006371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:15.110466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff15d7-e58d-4281-858a-88e23fa165a9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:15.144636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:15.145197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:15.145479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:15.145749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:15.157733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:15.159182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:15.173434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:15.174509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:15.175421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:15.175421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:15.175421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:15.175421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 526/1047] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:15.181451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:15.285816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf835-9a8f-4639-b24e-ca692025cb0b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:15.288208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:15.324502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:15.325185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:15.325467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:15.326244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:15.326894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:15.327192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:15.328671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:15.343877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 522/1048] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:15.346806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:15.376766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:15.462752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57442156-0193-4f27-b82e-7f64cee54d5c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:15.498352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:15.499041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:15.499324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:15.499599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:15.506258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:15.507694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:52:15.519964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:15.520987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:15.521879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:15.523984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 527/1049] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:52:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:15.527234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:15.635714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36d9ee-7c67-4069-a60a-478e250d95c2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:15.671394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5abc0a-24f7-40d1-bb8e-ccb5a1e725f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:15.672034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5abc0a-24f7-40d1-bb8e-ccb5a1e725f9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:15.672314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5abc0a-24f7-40d1-bb8e-ccb5a1e725f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:15.672586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5abc0a-24f7-40d1-bb8e-ccb5a1e725f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:15.673268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5abc0a-24f7-40d1-bb8e-ccb5a1e725f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:15.673572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5abc0a-24f7-40d1-bb8e-ccb5a1e725f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:15.675000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5abc0a-24f7-40d1-bb8e-ccb5a1e725f9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:15.687748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 523/1050] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:15.693779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:15.786271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:15.802544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488275-6a66-4ee9-912d-e3e368f17c25 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:15.836266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc6bdb-4e4f-4343-a27d-d352261ab4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:15.836869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc6bdb-4e4f-4343-a27d-d352261ab4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:15.837152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc6bdb-4e4f-4343-a27d-d352261ab4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:15.837427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc6bdb-4e4f-4343-a27d-d352261ab4f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:15.838143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc6bdb-4e4f-4343-a27d-d352261ab4f8 None admin] RBAC: Adding query filter params (role.id=a1bebaf3d4f249b99aeb6ac4cfde207a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=91413e6647284487bae08bb7eb02da0b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:15.838452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc6bdb-4e4f-4343-a27d-d352261ab4f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:15.839903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc6bdb-4e4f-4343-a27d-d352261ab4f8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:15.851872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 528/1051] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:15 2022] GET /identity/v3/role_assignments?user.id=91413e6647284487bae08bb7eb02da0b&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=a1bebaf3d4f249b99aeb6ac4cfde207a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:16.603898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf38cea-678a-4423-97d5-e0afec041d48 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:16.604454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf38cea-678a-4423-97d5-e0afec041d48 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:16.604454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf38cea-678a-4423-97d5-e0afec041d48 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:16.605930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 524/1052] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:16.611588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8d49-5bc4-4f37-a83e-c55c47487f21 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:16.612110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8d49-5bc4-4f37-a83e-c55c47487f21 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:16.612196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8d49-5bc4-4f37-a83e-c55c47487f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:16.631208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f36d8d49-5bc4-4f37-a83e-c55c47487f21 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:16.632424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f36d8d49-5bc4-4f37-a83e-c55c47487f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:16.703956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 529/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:16.967936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf38cea-678a-4423-97d5-e0afec041d48 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:16.994635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:17.017005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260b82-8faf-4578-aaea-2ec498121300 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.017298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260b82-8faf-4578-aaea-2ec498121300 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.017573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260b82-8faf-4578-aaea-2ec498121300 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.018937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 530/1054] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:52:17.024527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4309-3089-4700-9a6f-66da82a4ae57 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.024813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4309-3089-4700-9a6f-66da82a4ae57 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.025086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4309-3089-4700-9a6f-66da82a4ae57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.051180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c71a4309-3089-4700-9a6f-66da82a4ae57 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:17.052845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c71a4309-3089-4700-9a6f-66da82a4ae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:17.077301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf38cea-678a-4423-97d5-e0afec041d48 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:17.115062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:17.115667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.115977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.116252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.130935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 531/1055] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:17.185974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:17.187416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:17.211932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:17.288907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 525/1056] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:17.294440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a4309-3089-4700-9a6f-66da82a4ae57 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:17.306318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:17.405494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:17.408016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a4309-3089-4700-9a6f-66da82a4ae57 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:17.443458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:17.444115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.444394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.444689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.456007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:17.457452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:17.473796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:17.474843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:17.475791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:17.475791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:17.475791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:17.475791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 532/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:17.480816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:17.509312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9331-5e73-40c9-a858-1f92860268e5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:17.540497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:17.547774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:17.553763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:17.554405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.554690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.554967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.565629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:17.570576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:17.571524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:17.572423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 526/1058] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:52:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:17.578442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:17.594763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4f586-d2e6-4e91-9e95-da6ff57ca604 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:17.631101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:17.632453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.632909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.633495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.634136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:17.634439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:17.635801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:17.649063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 533/1059] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:17.684074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eb44e-c46b-41e4-a9a9-b3c2f71768bf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:17.722430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:17.723061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.723345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.723620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.724342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:17.724638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:17.726028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:17.741260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 527/1060] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:52:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:17.742502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:17.854512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebf3ff-31a7-49c8-8617-74b3f78e36f8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:17.881598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:17.895242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959b140-24e8-4c5f-80a6-24108bae8e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:17.896077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959b140-24e8-4c5f-80a6-24108bae8e8d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.896188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959b140-24e8-4c5f-80a6-24108bae8e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.896463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959b140-24e8-4c5f-80a6-24108bae8e8d None admin] PATH_INFO: `/v3/users/52f2eba4ee7a416682907083011f283a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.912533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959b140-24e8-4c5f-80a6-24108bae8e8d None admin] RBAC: Authorizing `identity:get_user(user_id=52f2eba4ee7a416682907083011f283a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:17.914021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959b140-24e8-4c5f-80a6-24108bae8e8d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:17.915722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 534/1061] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:17 2022] GET /identity/v3/users/52f2eba4ee7a416682907083011f283a => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:17.922470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:17.974536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:17.996906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0fba3-9409-4b10-bb09-6d78ce86d0d2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:17.997184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0fba3-9409-4b10-bb09-6d78ce86d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:17.997523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0fba3-9409-4b10-bb09-6d78ce86d0d2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:17.998877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 535/1062] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:18.004203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c6e84-e2d4-4ebd-8295-7702a7d081d7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.004487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c6e84-e2d4-4ebd-8295-7702a7d081d7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.004765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c6e84-e2d4-4ebd-8295-7702a7d081d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.022975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ce5c6e84-e2d4-4ebd-8295-7702a7d081d7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:18.024164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ce5c6e84-e2d4-4ebd-8295-7702a7d081d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:18.031009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dffbbe-b4d9-4dae-8478-73ad5e01d1b2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:18.063855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:18.064465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.064745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.065010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.096144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 536/1063] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:18.132000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:18.133374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:18.177456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:18.235754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 528/1064] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:18.237030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c6e84-e2d4-4ebd-8295-7702a7d081d7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:18.333280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c6e84-e2d4-4ebd-8295-7702a7d081d7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:18.344370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:18.365435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:18.368282 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:18.368916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.369194 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.369467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.370585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] RBAC: Authorizing `identity:get_project(project_id=6020ddef123546d1a0795313125c4a64)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:18.372048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:18.374291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 537/1065] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:18 2022] GET /identity/v3/projects/6020ddef123546d1a0795313125c4a64 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:18.380062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:18.468756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77858dc1-3c53-438f-9ddd-2f27632217f4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:18.480623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94790d14-e0af-4498-92a8-47aaed93e2b3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:18.502854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:18.503484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.503763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.504064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.514015 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:18.514389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:18.514636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.514919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.515196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.515942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=6020ddef123546d1a0795313125c4a64, user.id=52f2eba4ee7a416682907083011f283a) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:18.516253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:18.517671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:18.519183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:18.520295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:18.521348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 529/1066] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Oct 6 10:52:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:18.526243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:18.533557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 538/1067] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:18 2022] GET /identity/v3/role_assignments?user.id=52f2eba4ee7a416682907083011f283a&scope.project.id=6020ddef123546d1a0795313125c4a64&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:18.536309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:18.633918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e451c-8bdf-451d-9404-dffcf53341d2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:18.670850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:18.671398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.671677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.672019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.674186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:18.674504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:18.675918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:18.684761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 530/1068] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Oct 6 10:52:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:18.690768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:18.735136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:18.790672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597162b6-8ea4-4b20-bee9-f94122424ad7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:18.822911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:18.823529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.823805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.824099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.834986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:18.836454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:18.850764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:18.851973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:18.852899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:18.852899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:18.852899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:18.852899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 539/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:18.857738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:18.923285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:18.956308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1271dac-c084-41f2-98c6-9b5b8864871b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:18.989124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:18.989727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:18.990003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:18.990271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:18.990902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:18.991201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:18.992569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:19.003981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 531/1070] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:19.009715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:19.101466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:19.108325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f6a6a-8769-4e19-a975-74a9a584aec1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:19.142034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:19.142659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:19.142943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:19.143219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:19.150860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:19.152336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:52:19.163702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:19.164669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:19.165969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 540/1071] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:52:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:19.170661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:19.269569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd64b37-bd0c-49af-83f6-5e858e1dcd34 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:19.295042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:19.303188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:19.303787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:19.304088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:19.304355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:19.305067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:19.305368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:19.306775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:19.323794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 532/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:52:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:19.330236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:19.433383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3258742-0117-4795-b37d-1df4f30b8d5b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:19.467228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:19.467900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:19.468185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:19.468462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:19.479226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:19.480705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:19.495434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:19.496586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:19.498287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:19.498287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:19.498287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:19.500118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 541/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:19.502830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:19.599332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc984f6-511a-420f-a401-27879f4f7d29 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:19.633243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:19.633835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:19.634111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:19.634378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:19.635001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:19.635299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:19.636662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:19.647602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 533/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:19.653311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:19.663082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:19.753261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28ba04-3876-4000-8313-77b0ce44485e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:19.785449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:19.786065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:19.786341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:19.786608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:19.792962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:19.794981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:52:19.805864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:19.806921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:19.807832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 542/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:19.813474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:19.863781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:19.917079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aa8b9-dd6e-4ebf-903a-7cdbb4c6ceac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:19.949853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d71800-4c82-4e44-ab59-5dc16831f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:19.950483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d71800-4c82-4e44-ab59-5dc16831f5db None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:19.951275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d71800-4c82-4e44-ab59-5dc16831f5db None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:19.951555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d71800-4c82-4e44-ab59-5dc16831f5db None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:19.952267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d71800-4c82-4e44-ab59-5dc16831f5db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:19.952576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d71800-4c82-4e44-ab59-5dc16831f5db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:19.953989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d71800-4c82-4e44-ab59-5dc16831f5db None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:19.967248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 534/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:19.973327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:20.037427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:20.072128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5172f23-1b29-4824-9512-682f4d81d5cb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:20.104664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f245c-f173-4f53-9f54-6b82cfbc1b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:20.105306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f245c-f173-4f53-9f54-6b82cfbc1b6d None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:20.105582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f245c-f173-4f53-9f54-6b82cfbc1b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:20.105853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f245c-f173-4f53-9f54-6b82cfbc1b6d None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/91413e6647284487bae08bb7eb02da0b/roles/a1bebaf3d4f249b99aeb6ac4cfde207a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:20.108506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f245c-f173-4f53-9f54-6b82cfbc1b6d None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=91413e6647284487bae08bb7eb02da0b, role_id=a1bebaf3d4f249b99aeb6ac4cfde207a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:20.109926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f245c-f173-4f53-9f54-6b82cfbc1b6d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:20.122207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 543/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:52:19 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/91413e6647284487bae08bb7eb02da0b/roles/a1bebaf3d4f249b99aeb6ac4cfde207a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:52:20.659576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58134f-8cc4-4ad9-b6e9-9cb034d22725 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:20.659827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58134f-8cc4-4ad9-b6e9-9cb034d22725 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:20.660127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58134f-8cc4-4ad9-b6e9-9cb034d22725 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:20.661473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 535/1078] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:20.666835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dfedd-c362-48a8-ba2e-4573ef739cde None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:20.667116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dfedd-c362-48a8-ba2e-4573ef739cde None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:20.667388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dfedd-c362-48a8-ba2e-4573ef739cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:20.686132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b34dfedd-c362-48a8-ba2e-4573ef739cde None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:20.687292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b34dfedd-c362-48a8-ba2e-4573ef739cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:20.752701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 544/1079] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:21.019751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58134f-8cc4-4ad9-b6e9-9cb034d22725 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:21.117021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58134f-8cc4-4ad9-b6e9-9cb034d22725 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:21.124575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:21.150082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:21.150697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:21.150972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:21.151239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:21.161171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:21.165864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:21.167043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:21.168015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 536/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:52:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:21.173512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34dfedd-c362-48a8-ba2e-4573ef739cde None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:21.226934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:21.273207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34dfedd-c362-48a8-ba2e-4573ef739cde None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:21.306928 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:21.307521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:21.307797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:21.308093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:21.308746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:21.309048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:21.310609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:21.318786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 545/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:52:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:21.324684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:21.425818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a12d1-8a87-458d-a2e1-f502c45c9119 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:21.459187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:21.459799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:21.460102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:21.460375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] PATH_INFO: `/v3/users/52f2eba4ee7a416682907083011f283a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:21.461472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] RBAC: Authorizing `identity:get_user(user_id=52f2eba4ee7a416682907083011f283a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:21.462885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:21.464483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 537/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:21 2022] GET /identity/v3/users/52f2eba4ee7a416682907083011f283a => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:21.470295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:21.568581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004ece1-355a-43ef-8a1e-e02d556a317b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:21.585283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:21.602069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d49027-f1d1-4089-9013-08001d61ed72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:21.602683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49027-f1d1-4089-9013-08001d61ed72 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:21.602959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49027-f1d1-4089-9013-08001d61ed72 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:21.603225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49027-f1d1-4089-9013-08001d61ed72 None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:21.604337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d49027-f1d1-4089-9013-08001d61ed72 None admin] RBAC: Authorizing `identity:get_project(project_id=6020ddef123546d1a0795313125c4a64)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:21.605756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d49027-f1d1-4089-9013-08001d61ed72 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:21.608224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 546/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:21 2022] GET /identity/v3/projects/6020ddef123546d1a0795313125c4a64 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:21.614200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:21.646642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:21.711689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3231a3-a4c0-4e15-8517-deca484e47af None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:21.745428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf6ba2-dab9-40f4-aaf0-07d1fd678835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:21.746074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf6ba2-dab9-40f4-aaf0-07d1fd678835 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:21.746350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf6ba2-dab9-40f4-aaf0-07d1fd678835 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:21.746621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf6ba2-dab9-40f4-aaf0-07d1fd678835 None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64/users/52f2eba4ee7a416682907083011f283a/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:21.748560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf6ba2-dab9-40f4-aaf0-07d1fd678835 None admin] RBAC: Authorizing `identity:create_grant(project_id=6020ddef123546d1a0795313125c4a64, user_id=52f2eba4ee7a416682907083011f283a, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:21.749970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf6ba2-dab9-40f4-aaf0-07d1fd678835 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:21.762449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 538/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:52:21 2022] PUT /identity/v3/projects/6020ddef123546d1a0795313125c4a64/users/52f2eba4ee7a416682907083011f283a/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:52:22.269522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811c73-ee46-440b-a491-da8f0e15ecf5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:22.270082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811c73-ee46-440b-a491-da8f0e15ecf5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:22.270082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811c73-ee46-440b-a491-da8f0e15ecf5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:22.271389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 547/1085] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:22.276622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576b9ed-c93a-4f6e-a605-a8c6400b9712 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:22.277046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576b9ed-c93a-4f6e-a605-a8c6400b9712 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:22.277299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576b9ed-c93a-4f6e-a605-a8c6400b9712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:22.297593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1576b9ed-c93a-4f6e-a605-a8c6400b9712 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:22.298779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1576b9ed-c93a-4f6e-a605-a8c6400b9712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:22.363311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 539/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:22.637680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811c73-ee46-440b-a491-da8f0e15ecf5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:22.739332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811c73-ee46-440b-a491-da8f0e15ecf5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:22.772500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:22.773112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:22.773391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:22.773664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:22.784427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:22.789236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:22.790348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:22.791345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 548/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Oct 6 10:52:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:22.796558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576b9ed-c93a-4f6e-a605-a8c6400b9712 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:22.820005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:22.893991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576b9ed-c93a-4f6e-a605-a8c6400b9712 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:22.926730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:22.927331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:22.927614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:22.927918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:22.928583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:22.928884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:22.930238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:22.938684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 540/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Oct 6 10:52:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:22.944465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:22.991305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:23.045234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993397f4-4d79-41d0-9735-a20ca98356b9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:23.079254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:23.079893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:23.080173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:23.080444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:23.090381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:23.092070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:23.106029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:23.107228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:23.108508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:23.108508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:23.108508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:23.112201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 549/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:23.113184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:23.196605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:23.212249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409435-8b62-4451-9e24-24eed176c722 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:23.245214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:23.245815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:23.246097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:23.246373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:23.247010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:23.247892 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:23.249250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:23.260573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 541/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:23.266431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:23.361403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:23.365214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56f366-92aa-4706-8e2c-7285e38d30b5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:23.399051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:23.399687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:23.400000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:23.400275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:23.408523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:23.409997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:52:23.422633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:23.423812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:23.425277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 550/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Oct 6 10:52:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:23.429528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:23.527127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513513d-4acd-48f1-8526-08b24b31e35f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:23.561441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:23.562043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:23.562326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:23.562599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:23.563320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:23.563626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:23.565085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:23.581651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 542/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Oct 6 10:52:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:23.587435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:23.687743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838e1a9-98c1-418f-9d0b-613941ab7b14 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:23.720961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30ed3d-13b8-4f41-a476-d8347db661b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:23.721567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30ed3d-13b8-4f41-a476-d8347db661b7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:23.721848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30ed3d-13b8-4f41-a476-d8347db661b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:23.722121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30ed3d-13b8-4f41-a476-d8347db661b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:23.749276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:23.790886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30ed3d-13b8-4f41-a476-d8347db661b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:23.792329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30ed3d-13b8-4f41-a476-d8347db661b7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:23.892647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 551/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 10:52:23.898411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:23.902161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:23.911813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a04e07-4355-4790-bf4f-0ce3f34f0554 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:23.912126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a04e07-4355-4790-bf4f-0ce3f34f0554 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:23.912402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a04e07-4355-4790-bf4f-0ce3f34f0554 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:23.913729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 552/1094] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:23.918586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173c3ce-543f-45b4-9eb1-68fdb4af40df None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:23.918866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173c3ce-543f-45b4-9eb1-68fdb4af40df None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:23.919715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173c3ce-543f-45b4-9eb1-68fdb4af40df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:23.937819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4173c3ce-543f-45b4-9eb1-68fdb4af40df None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:23.939008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4173c3ce-543f-45b4-9eb1-68fdb4af40df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:23.992417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:24.001827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115fad3-042c-4c23-8837-4803ba557ee5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.006430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 553/1095] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:24.036334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.036959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.037237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.037508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.047275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.048670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:24.062684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:24.063659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:52:24.064623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:24.064623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:52:24.064623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:24.064623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 543/1096] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:24.069435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173c3ce-543f-45b4-9eb1-68fdb4af40df None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.167506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173c3ce-543f-45b4-9eb1-68fdb4af40df None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.201235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.201833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.202115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.202390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.203033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:24.203336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.204732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:24.215802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 554/1097] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:24.221948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.277761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:24.279520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.323534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a11a14-d1fc-42e9-8462-899d6cf09c80 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.330244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:24.357951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.358580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.358855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.359124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.365534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.366953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:52:24.380094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:24.381247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:52:24.382206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:24.385708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433a298-e4be-435d-bbcd-13d1aa415ce3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.390063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 544/1098] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:24.391033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.424784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.425410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.425740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.426039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.437219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:24.441948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:24.442980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:24.444339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 555/1099] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:52:24.449014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.518103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6df47-4136-478b-8c01-aeee19acfb96 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.535912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:24.555111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.555709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.556013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.556283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.556969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:24.557268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.558502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede64df-7de7-428b-a41a-c05ed373aa53 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.558680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:24.568392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:24.571414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 545/1100] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:24.578205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.596190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.596804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.597086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.597364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.598057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:24.598361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.599737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:24.608964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 556/1101] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:24.614793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.683678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15880-3725-4412-b5cf-6536dc2f7560 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.720208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.720817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.721102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.721378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.722103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] RBAC: Adding query filter params (role.id=a1bebaf3d4f249b99aeb6ac4cfde207a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=91413e6647284487bae08bb7eb02da0b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:24.722412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.723865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:24.724780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6298a-59b7-4f70-9c30-11f8064f30fd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.729499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 546/1102] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/role_assignments?user.id=91413e6647284487bae08bb7eb02da0b&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=a1bebaf3d4f249b99aeb6ac4cfde207a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:24.766851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.767504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.767784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.768088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] PATH_INFO: `/v3/users/52f2eba4ee7a416682907083011f283a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.769284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] RBAC: Authorizing `identity:get_user(user_id=52f2eba4ee7a416682907083011f283a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.770756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:24.772450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 557/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/users/52f2eba4ee7a416682907083011f283a => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:52:24.778772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:24.823823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:24.884264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de70d3f-67e2-4791-ab7b-64c6d517ab0c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:24.919728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:24.920499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:24.920771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:24.921036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:24.989058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:24.990516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:25.032490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:25.092444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 547/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 10:52:25.098621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:25.139369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:25.201434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf35af8-0329-4911-908b-23777e14b82c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:25.235626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ffc60-538b-4469-b1a7-e76b7bd9a89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:25.236293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ffc60-538b-4469-b1a7-e76b7bd9a89a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:25.236577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ffc60-538b-4469-b1a7-e76b7bd9a89a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:25.236849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ffc60-538b-4469-b1a7-e76b7bd9a89a None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:25.237956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ffc60-538b-4469-b1a7-e76b7bd9a89a None admin] RBAC: Authorizing `identity:get_project(project_id=6020ddef123546d1a0795313125c4a64)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:25.239381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ffc60-538b-4469-b1a7-e76b7bd9a89a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:25.241053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 558/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:25 2022] GET /identity/v3/projects/6020ddef123546d1a0795313125c4a64 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:25.247828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:25.364292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731077cd-5ef2-43f0-91af-5eee4de4fa72 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:25.399327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f63359-84d4-4d9b-9990-0f17f4887ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:25.400181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f63359-84d4-4d9b-9990-0f17f4887ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:25.400633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f63359-84d4-4d9b-9990-0f17f4887ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:25.401075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f63359-84d4-4d9b-9990-0f17f4887ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:25.401808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f63359-84d4-4d9b-9990-0f17f4887ae6 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=6020ddef123546d1a0795313125c4a64, user.id=52f2eba4ee7a416682907083011f283a) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:25.402118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f63359-84d4-4d9b-9990-0f17f4887ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:25.403830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f63359-84d4-4d9b-9990-0f17f4887ae6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:25.410173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 548/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:25 2022] GET /identity/v3/role_assignments?user.id=52f2eba4ee7a416682907083011f283a&scope.project.id=6020ddef123546d1a0795313125c4a64&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:27.603505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c052f-ffaa-461c-9947-03547c68d091 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:27.604058 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c052f-ffaa-461c-9947-03547c68d091 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:27.604171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c052f-ffaa-461c-9947-03547c68d091 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:27.605474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 559/1107] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:27.610950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dd58f-f9d4-4e92-8523-14c36dacdcd3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:27.611387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dd58f-f9d4-4e92-8523-14c36dacdcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:27.611506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dd58f-f9d4-4e92-8523-14c36dacdcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:27.618758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:27.631486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b74dd58f-f9d4-4e92-8523-14c36dacdcd3 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:27.632698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b74dd58f-f9d4-4e92-8523-14c36dacdcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:27.697361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 549/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:27.967149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c052f-ffaa-461c-9947-03547c68d091 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:28.066887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c052f-ffaa-461c-9947-03547c68d091 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:28.074204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:28.101837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef99a6-a06b-4004-bdae-f6dc644e0f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:28.102444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef99a6-a06b-4004-bdae-f6dc644e0f1b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:28.102720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef99a6-a06b-4004-bdae-f6dc644e0f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:28.102996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef99a6-a06b-4004-bdae-f6dc644e0f1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:28.104111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef99a6-a06b-4004-bdae-f6dc644e0f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:28.105532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef99a6-a06b-4004-bdae-f6dc644e0f1b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:28.107746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 560/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:28.113488 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dd58f-f9d4-4e92-8523-14c36dacdcd3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:28.208002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dd58f-f9d4-4e92-8523-14c36dacdcd3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:28.241241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4ad07-e2ca-4ce3-9f3e-7e27e796de6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:28.241830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4ad07-e2ca-4ce3-9f3e-7e27e796de6d None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:28.242111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4ad07-e2ca-4ce3-9f3e-7e27e796de6d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:28.242381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4ad07-e2ca-4ce3-9f3e-7e27e796de6d None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:28.243019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4ad07-e2ca-4ce3-9f3e-7e27e796de6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:28.244469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4ad07-e2ca-4ce3-9f3e-7e27e796de6d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:28.252695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:28.266206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 550/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:52:28 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:52:30.452564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8b4f3-8da3-4287-a559-1b5180fdecab None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:30.453095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8b4f3-8da3-4287-a559-1b5180fdecab None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:30.453095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8b4f3-8da3-4287-a559-1b5180fdecab None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:30.454431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 561/1111] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:30.459685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3b158-5c26-4b5f-82e5-dd1f802700f5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:30.460122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3b158-5c26-4b5f-82e5-dd1f802700f5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:30.460266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3b158-5c26-4b5f-82e5-dd1f802700f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:30.477968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-25c3b158-5c26-4b5f-82e5-dd1f802700f5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:30.479126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25c3b158-5c26-4b5f-82e5-dd1f802700f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:30.543237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 551/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:30.812204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8b4f3-8da3-4287-a559-1b5180fdecab None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:30.911439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8b4f3-8da3-4287-a559-1b5180fdecab None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:30.945022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:30.945635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:30.945916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:30.946191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] PATH_INFO: `/v3/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:30.947572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] RBAC: Authorizing `identity:get_role(role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:30.948964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:30.950311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 562/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:30 2022] GET /identity/v3/roles/2372f2ee163c430ca4b943c833677192 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:30.956087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3b158-5c26-4b5f-82e5-dd1f802700f5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:31.054623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3b158-5c26-4b5f-82e5-dd1f802700f5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:31.074514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:31.089517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:31.090140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:31.090419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:31.091217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] PATH_INFO: `/v3/users/c828098635734b5084d0e8ae7dd8b2f1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:31.104323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] RBAC: Authorizing `identity:get_user(user_id=c828098635734b5084d0e8ae7dd8b2f1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:31.105770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:31.108501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 552/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:30 2022] GET /identity/v3/users/c828098635734b5084d0e8ae7dd8b2f1 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:31.114244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:31.157997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:31.214270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ee41-fc2e-4846-9f5f-545c8022856d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:31.247971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:31.248562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:31.248837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:31.249108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:31.313684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:31.315125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:31.354183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:31.410723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 563/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:31.416629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:31.511486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6933-62f8-4917-9df0-2dd1da89235f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:31.543294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890e90c-e39b-4401-a06c-05d5c494736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:31.544451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890e90c-e39b-4401-a06c-05d5c494736e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:31.544728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890e90c-e39b-4401-a06c-05d5c494736e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:31.544998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890e90c-e39b-4401-a06c-05d5c494736e None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:31.546087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890e90c-e39b-4401-a06c-05d5c494736e None admin] RBAC: Authorizing `identity:get_project(project_id=6020ddef123546d1a0795313125c4a64)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:31.547501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890e90c-e39b-4401-a06c-05d5c494736e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:31.549114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 553/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:31 2022] GET /identity/v3/projects/6020ddef123546d1a0795313125c4a64 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:31.555191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:31.650554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bc08a-cc27-452a-974c-df5774e137ff None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:31.683808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccfafd-1219-4788-9666-32814949f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:31.684992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccfafd-1219-4788-9666-32814949f514 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:31.685421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccfafd-1219-4788-9666-32814949f514 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:31.685698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccfafd-1219-4788-9666-32814949f514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:31.686406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccfafd-1219-4788-9666-32814949f514 None admin] RBAC: Adding query filter params (role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=6020ddef123546d1a0795313125c4a64, user.id=c828098635734b5084d0e8ae7dd8b2f1) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:31.686713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccfafd-1219-4788-9666-32814949f514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:31.688270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccfafd-1219-4788-9666-32814949f514 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:31.693468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 564/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:31 2022] GET /identity/v3/role_assignments?user.id=c828098635734b5084d0e8ae7dd8b2f1&scope.project.id=6020ddef123546d1a0795313125c4a64&role.id=2372f2ee163c430ca4b943c833677192 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:33.792756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb29d8-c3fe-4455-95b1-459e37ad0dc6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:33.793276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb29d8-c3fe-4455-95b1-459e37ad0dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:33.793276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb29d8-c3fe-4455-95b1-459e37ad0dc6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:33.795458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 554/1118] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:33.800751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd781e69-7446-4a0b-a8b4-a028bf3f4004 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:33.801173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd781e69-7446-4a0b-a8b4-a028bf3f4004 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:33.801303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd781e69-7446-4a0b-a8b4-a028bf3f4004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:33.814599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:33.820119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cd781e69-7446-4a0b-a8b4-a028bf3f4004 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:33.821307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cd781e69-7446-4a0b-a8b4-a028bf3f4004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:33.885616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 565/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:34.158520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb29d8-c3fe-4455-95b1-459e37ad0dc6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:34.204695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:34.256825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb29d8-c3fe-4455-95b1-459e37ad0dc6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:34.291058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:34.291675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:34.291979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:34.292248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] PATH_INFO: `/v3/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:34.293605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] RBAC: Authorizing `identity:get_role(role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:34.295652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:34.296988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 555/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:34 2022] GET /identity/v3/roles/2372f2ee163c430ca4b943c833677192 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:34.302834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd781e69-7446-4a0b-a8b4-a028bf3f4004 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:34.401765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd781e69-7446-4a0b-a8b4-a028bf3f4004 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:34.435687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:34.436268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:34.436546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:34.436815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] PATH_INFO: `/v3/users/c828098635734b5084d0e8ae7dd8b2f1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:34.437908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] RBAC: Authorizing `identity:get_user(user_id=c828098635734b5084d0e8ae7dd8b2f1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:34.439320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:34.440912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 566/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:34 2022] GET /identity/v3/users/c828098635734b5084d0e8ae7dd8b2f1 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:34.446818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:34.540751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cedda-94d2-4fde-a878-577a9163848a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:34.554039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:34.573352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526aba9-69c0-4225-b9bc-9bde3ce3467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:34.573964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526aba9-69c0-4225-b9bc-9bde3ce3467b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:34.574239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526aba9-69c0-4225-b9bc-9bde3ce3467b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:34.574505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526aba9-69c0-4225-b9bc-9bde3ce3467b None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:34.576231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526aba9-69c0-4225-b9bc-9bde3ce3467b None admin] RBAC: Authorizing `identity:get_project(project_id=6020ddef123546d1a0795313125c4a64)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:34.577634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526aba9-69c0-4225-b9bc-9bde3ce3467b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:34.579225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 556/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:34 2022] GET /identity/v3/projects/6020ddef123546d1a0795313125c4a64 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:34.585054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:34.588731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:34.682116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f682ad-42c0-43dc-994b-a92278d5c610 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:34.715216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139168a-3ce8-45e1-9716-5d9df842b327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:34.715973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139168a-3ce8-45e1-9716-5d9df842b327 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:34.716361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139168a-3ce8-45e1-9716-5d9df842b327 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:34.716638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139168a-3ce8-45e1-9716-5d9df842b327 None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64/users/c828098635734b5084d0e8ae7dd8b2f1/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:34.718754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139168a-3ce8-45e1-9716-5d9df842b327 None admin] RBAC: Authorizing `identity:create_grant(project_id=6020ddef123546d1a0795313125c4a64, user_id=c828098635734b5084d0e8ae7dd8b2f1, role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:34.720197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139168a-3ce8-45e1-9716-5d9df842b327 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:34.732947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 567/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:52:34 2022] PUT /identity/v3/projects/6020ddef123546d1a0795313125c4a64/users/c828098635734b5084d0e8ae7dd8b2f1/roles/2372f2ee163c430ca4b943c833677192 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:52:36.844722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa100-b399-4656-bb94-e4224d395d13 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:36.845251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa100-b399-4656-bb94-e4224d395d13 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:36.845251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa100-b399-4656-bb94-e4224d395d13 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:36.846596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 557/1124] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:36.851676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102fa7-f1ce-41b0-bc2f-ffe04b886841 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:36.852094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102fa7-f1ce-41b0-bc2f-ffe04b886841 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:36.852261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102fa7-f1ce-41b0-bc2f-ffe04b886841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:36.870426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e3102fa7-f1ce-41b0-bc2f-ffe04b886841 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:36.871585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3102fa7-f1ce-41b0-bc2f-ffe04b886841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:36.929223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:36.939159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 568/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:37.208562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa100-b399-4656-bb94-e4224d395d13 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:37.305232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa100-b399-4656-bb94-e4224d395d13 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:37.336631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:37.337242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:37.337517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:37.337784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] PATH_INFO: `/v3/roles/2372f2ee163c430ca4b943c833677192` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:37.339141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] RBAC: Authorizing `identity:get_role(role_id=2372f2ee163c430ca4b943c833677192)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:37.341035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:37.342313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 558/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:37 2022] GET /identity/v3/roles/2372f2ee163c430ca4b943c833677192 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:37.347998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102fa7-f1ce-41b0-bc2f-ffe04b886841 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:37.447464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102fa7-f1ce-41b0-bc2f-ffe04b886841 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:37.482790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:37.483410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:37.483692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:37.483995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] PATH_INFO: `/v3/users/c828098635734b5084d0e8ae7dd8b2f1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:37.485109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] RBAC: Authorizing `identity:get_user(user_id=c828098635734b5084d0e8ae7dd8b2f1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:37.486526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:37.488137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 569/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:37 2022] GET /identity/v3/users/c828098635734b5084d0e8ae7dd8b2f1 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:37.493861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:37.542151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:37.588965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef809-b958-4509-b3ee-1e86486bdf32 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:37.621002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:37.621592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:37.621897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:37.622164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:37.685503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:37.686917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:37.732586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:37.781004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 559/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:37.786772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:37.846382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:37.884764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0a5d-97b1-431f-bb96-5b400c109567 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:37.917093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c251b2e-7aa9-4840-9dea-6ababaab6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:37.917700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c251b2e-7aa9-4840-9dea-6ababaab6505 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:37.917976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c251b2e-7aa9-4840-9dea-6ababaab6505 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:37.918243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c251b2e-7aa9-4840-9dea-6ababaab6505 None admin] PATH_INFO: `/v3/projects/6020ddef123546d1a0795313125c4a64` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:37.919486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c251b2e-7aa9-4840-9dea-6ababaab6505 None admin] RBAC: Authorizing `identity:get_project(project_id=6020ddef123546d1a0795313125c4a64)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:37.920924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c251b2e-7aa9-4840-9dea-6ababaab6505 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:37.922556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 570/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:37 2022] GET /identity/v3/projects/6020ddef123546d1a0795313125c4a64 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:37.928573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:38.023409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006596b-f28b-4a9c-85dc-03a5a53536d9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:38.055886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac407cc-e28e-40ac-a024-cd227717222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:38.056434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac407cc-e28e-40ac-a024-cd227717222d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:38.056713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac407cc-e28e-40ac-a024-cd227717222d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:38.056986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac407cc-e28e-40ac-a024-cd227717222d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:38.057700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac407cc-e28e-40ac-a024-cd227717222d None admin] RBAC: Adding query filter params (role.id=2372f2ee163c430ca4b943c833677192, scope.project.id=6020ddef123546d1a0795313125c4a64, user.id=c828098635734b5084d0e8ae7dd8b2f1) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:38.058003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac407cc-e28e-40ac-a024-cd227717222d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:38.060019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac407cc-e28e-40ac-a024-cd227717222d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:38.065516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 560/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:37 2022] GET /identity/v3/role_assignments?user.id=c828098635734b5084d0e8ae7dd8b2f1&scope.project.id=6020ddef123546d1a0795313125c4a64&role.id=2372f2ee163c430ca4b943c833677192 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:40.255013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db681-2bf0-43fb-b59e-911c29b6cf2c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:40.255537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db681-2bf0-43fb-b59e-911c29b6cf2c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:40.255537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db681-2bf0-43fb-b59e-911c29b6cf2c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:40.256933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 571/1131] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:40.262091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a43442-1ab4-45a7-a180-1926e033071c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:40.262715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a43442-1ab4-45a7-a180-1926e033071c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:40.262876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a43442-1ab4-45a7-a180-1926e033071c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:40.281924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e3a43442-1ab4-45a7-a180-1926e033071c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:40.283105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e3a43442-1ab4-45a7-a180-1926e033071c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:40.292074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:40.352389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 561/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:40.623291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05db681-2bf0-43fb-b59e-911c29b6cf2c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:40.723483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05db681-2bf0-43fb-b59e-911c29b6cf2c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:40.750769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:40.756603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b7c55-8430-4685-9569-b5089948cfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:40.757221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b7c55-8430-4685-9569-b5089948cfc3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:40.757504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b7c55-8430-4685-9569-b5089948cfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:40.757779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b7c55-8430-4685-9569-b5089948cfc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:40.758886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b7c55-8430-4685-9569-b5089948cfc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:40.760346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b7c55-8430-4685-9569-b5089948cfc3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:40.761907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 572/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:40.768180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a43442-1ab4-45a7-a180-1926e033071c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:40.862835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a43442-1ab4-45a7-a180-1926e033071c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:40.895896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2463cc-b380-4134-805a-abc297f0a4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:40.896436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2463cc-b380-4134-805a-abc297f0a4cc None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:40.896711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2463cc-b380-4134-805a-abc297f0a4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:40.896978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2463cc-b380-4134-805a-abc297f0a4cc None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:40.897618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2463cc-b380-4134-805a-abc297f0a4cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:40.899017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2463cc-b380-4134-805a-abc297f0a4cc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:40.914082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:40.927191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 562/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:52:40 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:52:43.143635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7f26a-80fc-4c76-931f-662f509281ad None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:43.144225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7f26a-80fc-4c76-931f-662f509281ad None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:43.144225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7f26a-80fc-4c76-931f-662f509281ad None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:43.145535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 573/1135] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:43.150961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d2f98-0aee-4612-a866-7107e635b073 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:43.151343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d2f98-0aee-4612-a866-7107e635b073 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:43.151514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d2f98-0aee-4612-a866-7107e635b073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:43.170682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6b6d2f98-0aee-4612-a866-7107e635b073 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:43.171884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6b6d2f98-0aee-4612-a866-7107e635b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:43.236670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 563/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:43.511450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7f26a-80fc-4c76-931f-662f509281ad None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:43.609793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7f26a-80fc-4c76-931f-662f509281ad None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:43.642297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a35e8-f977-473a-97d5-c049c5f554eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:43.642905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a35e8-f977-473a-97d5-c049c5f554eb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:43.643182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a35e8-f977-473a-97d5-c049c5f554eb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:43.643453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a35e8-f977-473a-97d5-c049c5f554eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:43.644557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a35e8-f977-473a-97d5-c049c5f554eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:43.645972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a35e8-f977-473a-97d5-c049c5f554eb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:43.647495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 574/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:43.653782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d2f98-0aee-4612-a866-7107e635b073 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:43.750979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d2f98-0aee-4612-a866-7107e635b073 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:43.775384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:43.785180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be99f53-a8ab-4014-b3f2-da453835b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:43.785786 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99f53-a8ab-4014-b3f2-da453835b542 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:43.786069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99f53-a8ab-4014-b3f2-da453835b542 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:43.786345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99f53-a8ab-4014-b3f2-da453835b542 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:43.787004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be99f53-a8ab-4014-b3f2-da453835b542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:43.788474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be99f53-a8ab-4014-b3f2-da453835b542 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:43.808439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 564/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:52:43 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:52:46.053919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447cee9-c75b-410f-aa89-9c0d7a726adf None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:46.054457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447cee9-c75b-410f-aa89-9c0d7a726adf None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:46.054457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447cee9-c75b-410f-aa89-9c0d7a726adf None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:46.055796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 575/1139] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:46.061174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16e4a7-aad7-4b7f-96b9-ffb5761288cd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:46.061593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16e4a7-aad7-4b7f-96b9-ffb5761288cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:46.061728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16e4a7-aad7-4b7f-96b9-ffb5761288cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:46.080361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2d16e4a7-aad7-4b7f-96b9-ffb5761288cd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:46.081518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2d16e4a7-aad7-4b7f-96b9-ffb5761288cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:46.146069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 565/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:46.417858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447cee9-c75b-410f-aa89-9c0d7a726adf None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:46.482551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:46.516479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447cee9-c75b-410f-aa89-9c0d7a726adf None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:46.549157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:46.549763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:46.550040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:46.550309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:46.560727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:46.565393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:46.566491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:46.567361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 576/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:52:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:46.572543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16e4a7-aad7-4b7f-96b9-ffb5761288cd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:46.668830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16e4a7-aad7-4b7f-96b9-ffb5761288cd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:46.681425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:46.703734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:46.704371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:46.704645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:46.704914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:46.705568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:46.705866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:46.707219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:46.714902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 566/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:52:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:46.721635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:46.819113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831eb518-95ab-49de-80af-eeaab54be654 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:46.844904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:46.853762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:46.854371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:46.854652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:46.854926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] PATH_INFO: `/v3/users/fdef962b24334d7b9a338eca18192d39` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:46.867706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] RBAC: Authorizing `identity:get_user(user_id=fdef962b24334d7b9a338eca18192d39)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:46.869181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:46.870751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 577/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:46 2022] GET /identity/v3/users/fdef962b24334d7b9a338eca18192d39 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:46.877287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:46.976953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4e54-4467-4904-a16c-495441b171c6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:47.008765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:47.009354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:47.009629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:47.009899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:47.050352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:47.075323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:47.076783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:47.168870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 567/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:47.174881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:47.271538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d24e4-fa32-42da-b886-5571bbbe27c3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:47.304296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e54bf-f000-4471-9988-83770fd316a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:47.304902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e54bf-f000-4471-9988-83770fd316a5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:47.305180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e54bf-f000-4471-9988-83770fd316a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:47.305450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e54bf-f000-4471-9988-83770fd316a5 None admin] PATH_INFO: `/v3/projects/1ef0f7a136b84feba8a49323695f13ca` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:47.306537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e54bf-f000-4471-9988-83770fd316a5 None admin] RBAC: Authorizing `identity:get_project(project_id=1ef0f7a136b84feba8a49323695f13ca)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:47.307979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e54bf-f000-4471-9988-83770fd316a5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:47.310153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 578/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:47 2022] GET /identity/v3/projects/1ef0f7a136b84feba8a49323695f13ca => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:47.316266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:47.388097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:47.412857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86677ac5-c40a-44f8-b4eb-c5087f40600a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:47.446071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ebcc5-e54e-4d54-a9ca-c6b42e0c8b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:47.446661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ebcc5-e54e-4d54-a9ca-c6b42e0c8b3c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:47.446936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ebcc5-e54e-4d54-a9ca-c6b42e0c8b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:47.447207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ebcc5-e54e-4d54-a9ca-c6b42e0c8b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:47.447946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ebcc5-e54e-4d54-a9ca-c6b42e0c8b3c None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1ef0f7a136b84feba8a49323695f13ca, user.id=fdef962b24334d7b9a338eca18192d39) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:47.448254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ebcc5-e54e-4d54-a9ca-c6b42e0c8b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:47.449754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ebcc5-e54e-4d54-a9ca-c6b42e0c8b3c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:47.454958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 568/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:47 2022] GET /identity/v3/role_assignments?user.id=fdef962b24334d7b9a338eca18192d39&scope.project.id=1ef0f7a136b84feba8a49323695f13ca&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:49.555370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28f9f-5602-405e-b1e5-972761402924 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:49.555915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28f9f-5602-405e-b1e5-972761402924 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:49.555915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28f9f-5602-405e-b1e5-972761402924 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:49.557261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 579/1147] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:49.563077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb58a1-aced-478c-b234-64eeb16c583b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:49.563451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb58a1-aced-478c-b234-64eeb16c583b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:49.563627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb58a1-aced-478c-b234-64eeb16c583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:49.582169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c7fb58a1-aced-478c-b234-64eeb16c583b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:49.583341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c7fb58a1-aced-478c-b234-64eeb16c583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:49.648107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 569/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:49.917183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c28f9f-5602-405e-b1e5-972761402924 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:49.969338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:50.016099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c28f9f-5602-405e-b1e5-972761402924 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:50.048514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:50.049129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:50.049409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:50.049679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:50.060088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:50.064840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:50.065917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:50.067039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 580/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:52:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:50.071954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb58a1-aced-478c-b234-64eeb16c583b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:50.134707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:50.167247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb58a1-aced-478c-b234-64eeb16c583b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:50.199288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:50.200533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:50.200811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:50.201077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:50.201723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:50.202019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:50.203354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:50.210954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 570/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:52:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:50.216805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:50.314280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9c5d0-d2ba-4c94-8df0-0c6051a51b50 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:50.326985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:50.347682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:50.348321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:50.348603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:50.348876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] PATH_INFO: `/v3/users/fdef962b24334d7b9a338eca18192d39` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:50.349975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=fdef962b24334d7b9a338eca18192d39)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:50.351398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:50.352984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 581/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:50 2022] GET /identity/v3/users/fdef962b24334d7b9a338eca18192d39 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:50.359356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:50.467133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425da8e-f460-4153-aca6-65797670ae36 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:50.494155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:50.501010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb298708-ff4a-4e55-a54d-3345f326e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:50.501632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298708-ff4a-4e55-a54d-3345f326e046 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:50.501909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298708-ff4a-4e55-a54d-3345f326e046 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:50.502181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298708-ff4a-4e55-a54d-3345f326e046 None admin] PATH_INFO: `/v3/projects/1ef0f7a136b84feba8a49323695f13ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:50.503292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb298708-ff4a-4e55-a54d-3345f326e046 None admin] RBAC: Authorizing `identity:get_project(project_id=1ef0f7a136b84feba8a49323695f13ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:50.504745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb298708-ff4a-4e55-a54d-3345f326e046 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:50.506349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 571/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:50 2022] GET /identity/v3/projects/1ef0f7a136b84feba8a49323695f13ca => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:50.512330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:50.612170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4eb801-e1c7-43ac-baa9-7e481b57e9d7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:50.644585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b718d9-80d5-474f-b2a5-f80ab4c45549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:50.645225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b718d9-80d5-474f-b2a5-f80ab4c45549 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:50.645506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b718d9-80d5-474f-b2a5-f80ab4c45549 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:50.645776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b718d9-80d5-474f-b2a5-f80ab4c45549 None admin] PATH_INFO: `/v3/projects/1ef0f7a136b84feba8a49323695f13ca/users/fdef962b24334d7b9a338eca18192d39/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:50.647629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b718d9-80d5-474f-b2a5-f80ab4c45549 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ef0f7a136b84feba8a49323695f13ca, user_id=fdef962b24334d7b9a338eca18192d39, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:50.649084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b718d9-80d5-474f-b2a5-f80ab4c45549 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:50.662730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 582/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:52:50 2022] PUT /identity/v3/projects/1ef0f7a136b84feba8a49323695f13ca/users/fdef962b24334d7b9a338eca18192d39/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:52:52.778528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0300729-5bf3-4c83-b173-9a2ab697d880 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:52.779088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0300729-5bf3-4c83-b173-9a2ab697d880 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:52.779088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0300729-5bf3-4c83-b173-9a2ab697d880 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:52.780430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 572/1154] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:52.785648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494b9da-4a74-4ace-841a-c19ec3ff96ad None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:52.786041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494b9da-4a74-4ace-841a-c19ec3ff96ad None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:52.786200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494b9da-4a74-4ace-841a-c19ec3ff96ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:52.804472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4494b9da-4a74-4ace-841a-c19ec3ff96ad None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:52.805651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4494b9da-4a74-4ace-841a-c19ec3ff96ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:52.839479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:52.873027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 583/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:53.143931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0300729-5bf3-4c83-b173-9a2ab697d880 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:53.241114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0300729-5bf3-4c83-b173-9a2ab697d880 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:53.273152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:53.273765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:53.274040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:53.274311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:53.285402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:52:53.290113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:53.291292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:52:53.292454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 573/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:53.297293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494b9da-4a74-4ace-841a-c19ec3ff96ad None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:53.392267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494b9da-4a74-4ace-841a-c19ec3ff96ad None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:53.427103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:53.427631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:53.427935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:53.428207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:53.428852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:53.429149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:53.430492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:53.438331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 584/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:52:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:53.444309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:53.500780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:53.542227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba243e0-faf3-4384-a739-824c583a87a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:53.573959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:53.575060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:53.575337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:53.575607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] PATH_INFO: `/v3/users/fdef962b24334d7b9a338eca18192d39` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:53.576778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] RBAC: Authorizing `identity:get_user(user_id=fdef962b24334d7b9a338eca18192d39)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:53.578194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:53.579745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 574/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:52:53 2022] GET /identity/v3/users/fdef962b24334d7b9a338eca18192d39 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:53.585508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:53.616563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:53.682993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cda00f-8cd5-4333-83b2-acfdbe7472aa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:53.715522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:53.716152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:53.716441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:53.716711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:53.781124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:53.782542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:53.810919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:53.876266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 585/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:53.882115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:53.978426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cbb08-f213-4cfb-b30d-21636422e1a0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:54.002474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:54.011227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d25a34-8922-461b-8029-61abfa442e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:54.012378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d25a34-8922-461b-8029-61abfa442e28 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:54.012729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d25a34-8922-461b-8029-61abfa442e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:54.013004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d25a34-8922-461b-8029-61abfa442e28 None admin] PATH_INFO: `/v3/projects/1ef0f7a136b84feba8a49323695f13ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:54.014095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d25a34-8922-461b-8029-61abfa442e28 None admin] RBAC: Authorizing `identity:get_project(project_id=1ef0f7a136b84feba8a49323695f13ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:54.015500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d25a34-8922-461b-8029-61abfa442e28 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:54.017113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 575/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:52:53 2022] GET /identity/v3/projects/1ef0f7a136b84feba8a49323695f13ca => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:54.023033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:54.118498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508c607-b0c9-46b4-a737-854f84acf3fb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:54.147250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:54.152429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c32939-de92-4440-a41e-941447cb2022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:54.153999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c32939-de92-4440-a41e-941447cb2022 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:54.154283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c32939-de92-4440-a41e-941447cb2022 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:54.154554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c32939-de92-4440-a41e-941447cb2022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:54.155262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c32939-de92-4440-a41e-941447cb2022 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1ef0f7a136b84feba8a49323695f13ca, user.id=fdef962b24334d7b9a338eca18192d39) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:54.155568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c32939-de92-4440-a41e-941447cb2022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:54.157009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c32939-de92-4440-a41e-941447cb2022 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:54.162625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 586/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:52:54 2022] GET /identity/v3/role_assignments?user.id=fdef962b24334d7b9a338eca18192d39&scope.project.id=1ef0f7a136b84feba8a49323695f13ca&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:56.314889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30d7f-0e5d-40d7-8c35-35708cfaf5e8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:56.315441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30d7f-0e5d-40d7-8c35-35708cfaf5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:56.315441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30d7f-0e5d-40d7-8c35-35708cfaf5e8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:56.316807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 576/1162] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:56.321988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2db50-f5ef-4ea0-b600-e2009e9328cb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:56.322385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2db50-f5ef-4ea0-b600-e2009e9328cb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:56.322540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2db50-f5ef-4ea0-b600-e2009e9328cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:56.342865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5ed2db50-f5ef-4ea0-b600-e2009e9328cb None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:56.344256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5ed2db50-f5ef-4ea0-b600-e2009e9328cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:56.410589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 587/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:52:56.674263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30d7f-0e5d-40d7-8c35-35708cfaf5e8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:56.775826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30d7f-0e5d-40d7-8c35-35708cfaf5e8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:56.807821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:56.808439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:56.809234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:56.809504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:56.866832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:56.875123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:56.876601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:56.972320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 577/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:52:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:52:56.979061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2db50-f5ef-4ea0-b600-e2009e9328cb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:57.075193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2db50-f5ef-4ea0-b600-e2009e9328cb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:57.099469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:57.108705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29128fd1-212c-4768-82f4-a06a26cbf342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:57.109320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29128fd1-212c-4768-82f4-a06a26cbf342 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:57.109594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29128fd1-212c-4768-82f4-a06a26cbf342 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:57.109886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29128fd1-212c-4768-82f4-a06a26cbf342 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:57.116307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29128fd1-212c-4768-82f4-a06a26cbf342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:57.117731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29128fd1-212c-4768-82f4-a06a26cbf342 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-29128fd1-212c-4768-82f4-a06a26cbf342 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:52:57.129339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:52:57.130600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:52:57.131479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 588/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:52:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:52:57.136363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:57.224546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:52:57.232152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e40076-703f-48c2-a7f0-f01a2806e625 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:57.265555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4653f00-2354-4079-a10c-a429346b6b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:57.266150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4653f00-2354-4079-a10c-a429346b6b35 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:57.266432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4653f00-2354-4079-a10c-a429346b6b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:57.266708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4653f00-2354-4079-a10c-a429346b6b35 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:57.267344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4653f00-2354-4079-a10c-a429346b6b35 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:52:57.267645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4653f00-2354-4079-a10c-a429346b6b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:57.269023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4653f00-2354-4079-a10c-a429346b6b35 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:57.276252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 578/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Oct 6 10:52:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:52:59.397614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed0ff3-2664-49db-9f9f-693abeaa91ef None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:59.398018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed0ff3-2664-49db-9f9f-693abeaa91ef None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:59.398165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed0ff3-2664-49db-9f9f-693abeaa91ef None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:59.399706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 589/1167] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:52:59.406480 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe297-10cd-4c35-aa37-1bed010ac352 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:59.406900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe297-10cd-4c35-aa37-1bed010ac352 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:59.407023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe297-10cd-4c35-aa37-1bed010ac352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:59.424638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-93cbe297-10cd-4c35-aa37-1bed010ac352 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:52:59.425800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93cbe297-10cd-4c35-aa37-1bed010ac352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:52:59.491692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 579/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:52:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:52:59.758189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed0ff3-2664-49db-9f9f-693abeaa91ef None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:52:59.856673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed0ff3-2664-49db-9f9f-693abeaa91ef None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:52:59.890055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f4e6f-f2d4-4d51-92d4-9ff1f3f5ca1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:52:59.890640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4e6f-f2d4-4d51-92d4-9ff1f3f5ca1d None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:52:59.890915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4e6f-f2d4-4d51-92d4-9ff1f3f5ca1d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:52:59.891181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4e6f-f2d4-4d51-92d4-9ff1f3f5ca1d None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:52:59.891867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f4e6f-f2d4-4d51-92d4-9ff1f3f5ca1d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:52:59.893221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f4e6f-f2d4-4d51-92d4-9ff1f3f5ca1d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:52:59.909805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 590/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Thu Oct 6 10:52:59 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:53:02.087050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc43300-9fbe-4922-9b54-098d28ffdec3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:02.087580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc43300-9fbe-4922-9b54-098d28ffdec3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:02.087580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc43300-9fbe-4922-9b54-098d28ffdec3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:02.088959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 580/1170] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:53:02.094353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d747b-08c0-4343-aa7e-38eadf808234 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:02.094751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d747b-08c0-4343-aa7e-38eadf808234 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:02.094911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d747b-08c0-4343-aa7e-38eadf808234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:02.112700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-500d747b-08c0-4343-aa7e-38eadf808234 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:53:02.113868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-500d747b-08c0-4343-aa7e-38eadf808234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:53:02.155966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:02.181259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 591/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:53:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:53:02.449873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc43300-9fbe-4922-9b54-098d28ffdec3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:02.544798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:02.557413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc43300-9fbe-4922-9b54-098d28ffdec3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:02.591819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce156d-5f15-4bcb-aa1d-2a9a7049929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:02.592460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce156d-5f15-4bcb-aa1d-2a9a7049929b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:02.592741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce156d-5f15-4bcb-aa1d-2a9a7049929b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:02.593017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce156d-5f15-4bcb-aa1d-2a9a7049929b None admin] PATH_INFO: `/v3/domains/55108f0803f24e25ba7eca511d99062b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:02.603029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce156d-5f15-4bcb-aa1d-2a9a7049929b None admin] RBAC: Authorizing `identity:get_domain(domain_id=55108f0803f24e25ba7eca511d99062b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:02.604508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce156d-5f15-4bcb-aa1d-2a9a7049929b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:02.606839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 581/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Oct 6 10:53:02 2022] GET /identity/v3/domains/55108f0803f24e25ba7eca511d99062b => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:02.612835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d747b-08c0-4343-aa7e-38eadf808234 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:02.707893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d747b-08c0-4343-aa7e-38eadf808234 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:02.740638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfea35-973d-4efe-9dca-4520fde01bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:02.741227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfea35-973d-4efe-9dca-4520fde01bf8 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:02.741503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfea35-973d-4efe-9dca-4520fde01bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:02.741771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfea35-973d-4efe-9dca-4520fde01bf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:02.742413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfea35-973d-4efe-9dca-4520fde01bf8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:02.743867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfea35-973d-4efe-9dca-4520fde01bf8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:02.768865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 592/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:53:02 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 10:53:04.953625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955abcc-a710-47f2-a282-2e528a8638f8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:04.954181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955abcc-a710-47f2-a282-2e528a8638f8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:04.954181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955abcc-a710-47f2-a282-2e528a8638f8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:04.955513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 582/1174] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:53:04.960799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefb2d7-0df3-4134-af4b-3c401e3e23a5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:04.961182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefb2d7-0df3-4134-af4b-3c401e3e23a5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:04.961348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefb2d7-0df3-4134-af4b-3c401e3e23a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:04.980029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1cefb2d7-0df3-4134-af4b-3c401e3e23a5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:53:04.981223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1cefb2d7-0df3-4134-af4b-3c401e3e23a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:53:05.003520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:05.048189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 593/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:53:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 10:53:05.316606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955abcc-a710-47f2-a282-2e528a8638f8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:05.413344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955abcc-a710-47f2-a282-2e528a8638f8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:05.445915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d483b-b468-4b7e-a8f3-cc568d69bdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:05.446533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d483b-b468-4b7e-a8f3-cc568d69bdae None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:05.446808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d483b-b468-4b7e-a8f3-cc568d69bdae None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:05.447077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d483b-b468-4b7e-a8f3-cc568d69bdae None admin] PATH_INFO: `/v3/domains/55108f0803f24e25ba7eca511d99062b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:05.448198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d483b-b468-4b7e-a8f3-cc568d69bdae None admin] RBAC: Authorizing `identity:get_domain(domain_id=55108f0803f24e25ba7eca511d99062b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:05.449649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d483b-b468-4b7e-a8f3-cc568d69bdae None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:05.451180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 583/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Oct 6 10:53:05 2022] GET /identity/v3/domains/55108f0803f24e25ba7eca511d99062b => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:05.456934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefb2d7-0df3-4134-af4b-3c401e3e23a5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:05.553077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefb2d7-0df3-4134-af4b-3c401e3e23a5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:05.585686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0e49d-5b1c-4cf6-8f2c-800fdaa660f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:05.586217 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0e49d-5b1c-4cf6-8f2c-800fdaa660f7 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:05.586492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0e49d-5b1c-4cf6-8f2c-800fdaa660f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:05.586761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0e49d-5b1c-4cf6-8f2c-800fdaa660f7 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:05.587398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0e49d-5b1c-4cf6-8f2c-800fdaa660f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:05.588896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0e49d-5b1c-4cf6-8f2c-800fdaa660f7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:05.611722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 594/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:53:05 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:53:07.830295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cac37-aa89-4f19-a016-ab7ab25ec7fd None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:07.830835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cac37-aa89-4f19-a016-ab7ab25ec7fd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:07.830835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cac37-aa89-4f19-a016-ab7ab25ec7fd None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:07.832232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 584/1178] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:53:07.837426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6a03-9a6b-4221-b0fb-9736491a6e5f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:07.837848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6a03-9a6b-4221-b0fb-9736491a6e5f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:07.837970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6a03-9a6b-4221-b0fb-9736491a6e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:07.856787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-655f6a03-9a6b-4221-b0fb-9736491a6e5f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:53:07.857947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-655f6a03-9a6b-4221-b0fb-9736491a6e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:53:07.867012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:07.925506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 595/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:53:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:53:08.197358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cac37-aa89-4f19-a016-ab7ab25ec7fd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:08.215901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:08.297586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cac37-aa89-4f19-a016-ab7ab25ec7fd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:08.329864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:08.330474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:08.330753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:08.331024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:08.341769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:53:08.346503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:53:08.347625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:53:08.348639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 585/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:53:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:53:08.353793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f6a03-9a6b-4221-b0fb-9736491a6e5f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:08.454977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f6a03-9a6b-4221-b0fb-9736491a6e5f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:08.488723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:08.489313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:08.489592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:08.489864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:08.490517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:53:08.490819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:08.492202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:08.500587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 596/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:53:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:08.506610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:08.582699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:08.605814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feacfd-99e9-4dc2-b70c-4721d014d137 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:08.638605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:08.639222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:08.639501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:08.639772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] PATH_INFO: `/v3/users/d1ff740d7e9f448cbd55f1105db6a14d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:08.652323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] RBAC: Authorizing `identity:get_user(user_id=d1ff740d7e9f448cbd55f1105db6a14d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:08.654000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:08.655576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 586/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:53:08 2022] GET /identity/v3/users/d1ff740d7e9f448cbd55f1105db6a14d => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:08.661588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:08.670922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:08.759504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f61272-b231-4664-a574-9b4b16a216bd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:08.792813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:08.793560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:08.793843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:08.794119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:08.859566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:08.861052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:08.868560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:08.958054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 597/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:53:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:53:08.964160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:09.062349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64717621-4857-45f5-8fef-cbd2c45cb4a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:09.096797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfd03e-7615-4e12-b113-9cfd728cb918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:09.097407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd03e-7615-4e12-b113-9cfd728cb918 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:09.097685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd03e-7615-4e12-b113-9cfd728cb918 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:09.097957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd03e-7615-4e12-b113-9cfd728cb918 None admin] PATH_INFO: `/v3/projects/5b6aa6f9da374aba98bcaa34e2525336` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:09.099058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfd03e-7615-4e12-b113-9cfd728cb918 None admin] RBAC: Authorizing `identity:get_project(project_id=5b6aa6f9da374aba98bcaa34e2525336)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:09.100500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfd03e-7615-4e12-b113-9cfd728cb918 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:09.102100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 587/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:53:08 2022] GET /identity/v3/projects/5b6aa6f9da374aba98bcaa34e2525336 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:09.108882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:09.205279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90420f78-74e8-4e17-94c9-5822d9d1f8b6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:09.212370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:09.239094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb6ecc-4d55-40ac-aff8-759e453a13c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:09.239676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb6ecc-4d55-40ac-aff8-759e453a13c9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:09.239986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb6ecc-4d55-40ac-aff8-759e453a13c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:09.240260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb6ecc-4d55-40ac-aff8-759e453a13c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:09.240971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb6ecc-4d55-40ac-aff8-759e453a13c9 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=5b6aa6f9da374aba98bcaa34e2525336, user.id=d1ff740d7e9f448cbd55f1105db6a14d) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:53:09.241276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb6ecc-4d55-40ac-aff8-759e453a13c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:09.242700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb6ecc-4d55-40ac-aff8-759e453a13c9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:09.248534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 598/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:53:09 2022] GET /identity/v3/role_assignments?user.id=d1ff740d7e9f448cbd55f1105db6a14d&scope.project.id=5b6aa6f9da374aba98bcaa34e2525336&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:11.376379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0e340-8228-48c5-a793-0b38b97ea573 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:11.376914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0e340-8228-48c5-a793-0b38b97ea573 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:11.376914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0e340-8228-48c5-a793-0b38b97ea573 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:11.378263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 588/1186] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:53:11.383481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ba6d7-5fae-442e-966c-0efa47987e44 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:11.383917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ba6d7-5fae-442e-966c-0efa47987e44 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:11.384063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ba6d7-5fae-442e-966c-0efa47987e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:11.402262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6c1ba6d7-5fae-442e-966c-0efa47987e44 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:53:11.403803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6c1ba6d7-5fae-442e-966c-0efa47987e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:53:11.469105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 599/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:53:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:53:11.738783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0e340-8228-48c5-a793-0b38b97ea573 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:11.741695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:11.838469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0e340-8228-48c5-a793-0b38b97ea573 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:11.872158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:11.872771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:11.873054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:11.873330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:11.883053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:53:11.888052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:53:11.889139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:53:11.890127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 589/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:53:11.895341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ba6d7-5fae-442e-966c-0efa47987e44 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:11.990534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:11.993468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ba6d7-5fae-442e-966c-0efa47987e44 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:12.026910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:12.028403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:12.028680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:12.028946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:12.029592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:53:12.029888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:12.031236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:12.039031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 600/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:12.045198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:12.104712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:12.142039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8ae0a-a1bf-42bb-a7bc-e3dbc0badad1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:12.175210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:12.175824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:12.176204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:12.176478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] PATH_INFO: `/v3/users/d1ff740d7e9f448cbd55f1105db6a14d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:12.177586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] RBAC: Authorizing `identity:get_user(user_id=d1ff740d7e9f448cbd55f1105db6a14d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:12.179005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:12.180597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 590/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:53:12 2022] GET /identity/v3/users/d1ff740d7e9f448cbd55f1105db6a14d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:12.186339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:12.283232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de983c-90f8-4459-ad9d-6f0882aa2afd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:12.316630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f68ba6-85dc-4ff6-9d77-0a9573ac0614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:12.317245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ba6-85dc-4ff6-9d77-0a9573ac0614 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:12.317529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ba6-85dc-4ff6-9d77-0a9573ac0614 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:12.317803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ba6-85dc-4ff6-9d77-0a9573ac0614 None admin] PATH_INFO: `/v3/projects/5b6aa6f9da374aba98bcaa34e2525336` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:12.318898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f68ba6-85dc-4ff6-9d77-0a9573ac0614 None admin] RBAC: Authorizing `identity:get_project(project_id=5b6aa6f9da374aba98bcaa34e2525336)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:12.320355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f68ba6-85dc-4ff6-9d77-0a9573ac0614 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:12.321956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 601/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:53:12 2022] GET /identity/v3/projects/5b6aa6f9da374aba98bcaa34e2525336 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:12.328212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:12.428246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f078457-7112-4fbe-bc9c-4b2db75d9a77 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:12.456328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:12.462422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5c09b-8a8b-4090-a998-850b6c85b555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:12.463068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5c09b-8a8b-4090-a998-850b6c85b555 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:12.463345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5c09b-8a8b-4090-a998-850b6c85b555 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:12.463619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5c09b-8a8b-4090-a998-850b6c85b555 None admin] PATH_INFO: `/v3/projects/5b6aa6f9da374aba98bcaa34e2525336/users/d1ff740d7e9f448cbd55f1105db6a14d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:12.465725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5c09b-8a8b-4090-a998-850b6c85b555 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b6aa6f9da374aba98bcaa34e2525336, user_id=d1ff740d7e9f448cbd55f1105db6a14d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:12.467136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5c09b-8a8b-4090-a998-850b6c85b555 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:12.478966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 591/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:53:12 2022] PUT /identity/v3/projects/5b6aa6f9da374aba98bcaa34e2525336/users/d1ff740d7e9f448cbd55f1105db6a14d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:53:14.613255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c10d21-9178-4ea2-85b7-e999e13d19b4 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:14.613800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c10d21-9178-4ea2-85b7-e999e13d19b4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:14.613800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c10d21-9178-4ea2-85b7-e999e13d19b4 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:14.615150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 602/1193] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:53:14.620444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f371-93a9-4b2f-a9a9-923ce85b33a1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:14.621352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f371-93a9-4b2f-a9a9-923ce85b33a1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:14.621512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f371-93a9-4b2f-a9a9-923ce85b33a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:14.640569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-39c2f371-93a9-4b2f-a9a9-923ce85b33a1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:53:14.641691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-39c2f371-93a9-4b2f-a9a9-923ce85b33a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:53:14.707362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 592/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:53:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:53:14.975648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c10d21-9178-4ea2-85b7-e999e13d19b4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:15.006446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:15.077518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c10d21-9178-4ea2-85b7-e999e13d19b4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:15.110447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:15.111061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:15.111345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:15.111623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:15.122962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:53:15.127753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:53:15.128894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:53:15.129792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 603/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:53:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:53:15.134944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f371-93a9-4b2f-a9a9-923ce85b33a1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:15.229529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f371-93a9-4b2f-a9a9-923ce85b33a1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:15.249680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:15.264236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:15.264828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:15.265103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:15.265372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:15.266027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:53:15.266325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:15.267679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:15.275404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 593/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:53:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:15.281584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:15.372569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:15.382686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8c6c9-2a24-4112-9bcd-36fd864eb8e0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:15.417150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:15.418568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:15.418850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:15.419122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] PATH_INFO: `/v3/users/d1ff740d7e9f448cbd55f1105db6a14d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:15.420272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] RBAC: Authorizing `identity:get_user(user_id=d1ff740d7e9f448cbd55f1105db6a14d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:15.421701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:15.423272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 604/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Oct 6 10:53:15 2022] GET /identity/v3/users/d1ff740d7e9f448cbd55f1105db6a14d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:15.428986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:15.523548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994d2d8-6a8d-41b7-93fd-3163d080afb5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:15.556287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:15.556874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:15.557151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:15.557424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:15.603704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:15.622275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:15.623700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:15.716710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 594/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:53:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:53:15.722625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:15.821098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd19f9-0424-46f1-a501-96ee1b831c71 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:15.854935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c363567-69b4-42fe-9d70-f165f9b90997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:15.855823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c363567-69b4-42fe-9d70-f165f9b90997 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:15.856135 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c363567-69b4-42fe-9d70-f165f9b90997 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:15.856409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c363567-69b4-42fe-9d70-f165f9b90997 None admin] PATH_INFO: `/v3/projects/5b6aa6f9da374aba98bcaa34e2525336` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:15.857516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c363567-69b4-42fe-9d70-f165f9b90997 None admin] RBAC: Authorizing `identity:get_project(project_id=5b6aa6f9da374aba98bcaa34e2525336)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:15.858933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c363567-69b4-42fe-9d70-f165f9b90997 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:15.860562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 605/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Oct 6 10:53:15 2022] GET /identity/v3/projects/5b6aa6f9da374aba98bcaa34e2525336 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:15.866565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:53:15.945150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:15.963792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a52d45-3b03-4528-bc8c-5c8e950d23ed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:53:15.997101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e692df1-6730-4083-bbb4-7595a23b0b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:53:15.997694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e692df1-6730-4083-bbb4-7595a23b0b5a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:53:15.997976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e692df1-6730-4083-bbb4-7595a23b0b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:53:15.998256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e692df1-6730-4083-bbb4-7595a23b0b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:53:15.998967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e692df1-6730-4083-bbb4-7595a23b0b5a None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=5b6aa6f9da374aba98bcaa34e2525336, user.id=d1ff740d7e9f448cbd55f1105db6a14d) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:53:15.999275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e692df1-6730-4083-bbb4-7595a23b0b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:53:16.000708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e692df1-6730-4083-bbb4-7595a23b0b5a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:53:16.006248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 595/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:53:15 2022] GET /identity/v3/role_assignments?user.id=d1ff740d7e9f448cbd55f1105db6a14d&scope.project.id=5b6aa6f9da374aba98bcaa34e2525336&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:53:25.850903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=87 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:53:26.003701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=27 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:11.750709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b2fc8-cd22-485b-a67e-9aa4776f0aff None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:11.751326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b2fc8-cd22-485b-a67e-9aa4776f0aff None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:11.751326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b2fc8-cd22-485b-a67e-9aa4776f0aff None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:11.752924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 606/1201] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:11.758172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7210-94e3-4c02-8788-1adab39fc755 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:11.758609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7210-94e3-4c02-8788-1adab39fc755 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:11.758758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7210-94e3-4c02-8788-1adab39fc755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:11.778744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-894b7210-94e3-4c02-8788-1adab39fc755 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:11.780007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-894b7210-94e3-4c02-8788-1adab39fc755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:11.871327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 596/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:12.153038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b2fc8-cd22-485b-a67e-9aa4776f0aff None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:12.293335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b2fc8-cd22-485b-a67e-9aa4776f0aff None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:12.339494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:12.340188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:12.340471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:12.340722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:12.352253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:12.353758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:12.370163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:12.371401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:12.373925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:12.373925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:12.373925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:12.373925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 607/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:12.378760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b7210-94e3-4c02-8788-1adab39fc755 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:12.513035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b7210-94e3-4c02-8788-1adab39fc755 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:12.544267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:12.550320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a28765-5471-422d-8864-789ab2749af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:12.550966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a28765-5471-422d-8864-789ab2749af0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:12.551531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a28765-5471-422d-8864-789ab2749af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:12.551897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a28765-5471-422d-8864-789ab2749af0 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:12.552698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a28765-5471-422d-8864-789ab2749af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:12.553079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a28765-5471-422d-8864-789ab2749af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:12.554686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a28765-5471-422d-8864-789ab2749af0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:12.569140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 597/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:12.575975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:12.638384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:12.711397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee43e30-6dd7-495b-b84a-64d07ea9ba9e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:12.759487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf10c5b-4c27-4388-abe0-89d9c0f9b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:12.760773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf10c5b-4c27-4388-abe0-89d9c0f9b55f None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:12.761166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf10c5b-4c27-4388-abe0-89d9c0f9b55f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:12.761453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf10c5b-4c27-4388-abe0-89d9c0f9b55f None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:12.762104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf10c5b-4c27-4388-abe0-89d9c0f9b55f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:12.763613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf10c5b-4c27-4388-abe0-89d9c0f9b55f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:12.794494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 608/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Oct 6 10:54:12 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:54:15.158547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5bd1-021e-4544-807e-4b6ce8c04d52 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:15.159096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5bd1-021e-4544-807e-4b6ce8c04d52 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:15.159197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5bd1-021e-4544-807e-4b6ce8c04d52 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:15.160515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 598/1206] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:15.166815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09751fb-34d2-465b-af4b-a420365964d5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:15.167434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09751fb-34d2-465b-af4b-a420365964d5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:15.167434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09751fb-34d2-465b-af4b-a420365964d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:15.187045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c09751fb-34d2-465b-af4b-a420365964d5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:15.188266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c09751fb-34d2-465b-af4b-a420365964d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:15.271187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 609/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:15.542182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b5bd1-021e-4544-807e-4b6ce8c04d52 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:15.653434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b5bd1-021e-4544-807e-4b6ce8c04d52 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:15.696349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:15.697038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:15.697296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:15.698124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:15.712832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:54:15.719806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:15.720881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:15.721734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 599/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:54:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:54:15.728412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09751fb-34d2-465b-af4b-a420365964d5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:15.781985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:15.856553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09751fb-34d2-465b-af4b-a420365964d5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:15.895437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:15.896093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:15.896404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:15.896801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:15.897441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:15.897777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:15.899404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:15.914114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 610/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:54:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:15.915133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:15.974397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:16.037717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10d130-53e4-4214-aacd-fefafc7ec4e2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:16.074470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:16.075107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:16.075386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:16.075726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:16.086606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:16.088584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:16.103066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:16.104864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:16.105795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:16.105795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:16.105795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:16.105795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 600/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:16.113414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:16.237489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:16.244196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c19c81-0e6d-4db9-b7d2-2b85de28a0bc None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:16.290321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:16.290989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:16.291452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:16.291811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:16.292564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:16.292893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:16.294354 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:16.309495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 611/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:16.314993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:16.441155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33469e45-a3ad-46a3-ae66-7a3a772a8869 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:16.443178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:16.485446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:16.486087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:16.486367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:16.486639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:16.498206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:16.499900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:54:16.512775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:16.515649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:16.516673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 601/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:16.520136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:16.644185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed904d-bfed-426e-8b5f-6f53dff6a819 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:16.691965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:16.692683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:16.692896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:16.693214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:16.694103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:16.694439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:16.696760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:16.716277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 612/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Oct 6 10:54:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:16.721730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:16.837558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0844d-38ea-47af-965a-9019619358b7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:16.875232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:16.875865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:16.876173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:16.876478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:16.918343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:16.960957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:16.962444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:17.075037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 602/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:54:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:54:17.080878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:17.102508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:17.194451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948137bd-4f19-43f0-8595-fa628f414b4b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:17.230697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:17.231335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:17.231611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:17.231910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:17.242326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:17.243806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:17.259694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:17.261333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:17.267023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:17.267023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:17.267023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:17.267023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 613/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:54:17.268337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:17.340969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:17.387352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94332447-d894-40c2-b3a7-909ef4d26fec None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:17.427531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:17.428236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:17.428556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:17.428737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:17.429442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:17.429810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:17.431426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:17.449804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 603/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:17.451305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:17.543617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:17.589835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558b55e-37f9-46be-ad01-58d8e3affe48 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:17.632430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:17.633144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:17.633483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:17.633717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:17.645492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:17.647227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:54:17.661971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:17.663006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:17.663977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 614/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:17.670994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:17.786924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f330-d172-4ea9-8bd9-0369cd481241 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:17.814256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:17.828682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ffb28-af1b-4042-89c7-e57e1419081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:17.829297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ffb28-af1b-4042-89c7-e57e1419081d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:17.829576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ffb28-af1b-4042-89c7-e57e1419081d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:17.829846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ffb28-af1b-4042-89c7-e57e1419081d None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:17.830562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ffb28-af1b-4042-89c7-e57e1419081d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:17.830841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ffb28-af1b-4042-89c7-e57e1419081d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:17.832289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ffb28-af1b-4042-89c7-e57e1419081d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:17.846505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 604/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:54:17.853225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:17.964372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf9bef-11b8-43ef-b60f-ed6ead17a1b2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:17.996621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:18.003792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d317fe-7529-4cb3-85e8-31376d9771d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:18.004498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d317fe-7529-4cb3-85e8-31376d9771d7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:18.004807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d317fe-7529-4cb3-85e8-31376d9771d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:18.005090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d317fe-7529-4cb3-85e8-31376d9771d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:18.005862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d317fe-7529-4cb3-85e8-31376d9771d7 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=49cbfb2f4ddb42c0b299311615c7b0dc) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:18.006172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d317fe-7529-4cb3-85e8-31376d9771d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:18.007890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d317fe-7529-4cb3-85e8-31376d9771d7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:18.014820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 615/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:54:17 2022] GET /identity/v3/role_assignments?user.id=49cbfb2f4ddb42c0b299311615c7b0dc&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:20.274966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776676ab-e087-4933-9b6e-eeb3ad1efb3a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:20.276132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776676ab-e087-4933-9b6e-eeb3ad1efb3a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:20.276260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776676ab-e087-4933-9b6e-eeb3ad1efb3a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:20.277951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 605/1220] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:20.283235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aff283-70eb-41a5-9d0f-8094bc766000 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:20.283683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aff283-70eb-41a5-9d0f-8094bc766000 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:20.283788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aff283-70eb-41a5-9d0f-8094bc766000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:20.302841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-75aff283-70eb-41a5-9d0f-8094bc766000 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:20.304078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-75aff283-70eb-41a5-9d0f-8094bc766000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:20.389080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 616/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:20.657185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776676ab-e087-4933-9b6e-eeb3ad1efb3a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:20.772415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776676ab-e087-4933-9b6e-eeb3ad1efb3a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:20.809942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:20.810653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:20.810950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:20.811263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:20.821611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:54:20.827091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:20.831917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:20.833114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 606/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:54:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:20.834342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aff283-70eb-41a5-9d0f-8094bc766000 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:20.947197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aff283-70eb-41a5-9d0f-8094bc766000 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:20.973408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:20.986319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:20.986979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:20.987315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:20.987615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:20.988420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:20.988732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:20.990344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:21.000246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 617/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:21.006266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:21.061173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:21.120872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5ec1d-bea9-4b0f-b1c1-ef960bc5d500 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:21.157326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:21.158025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:21.158311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:21.158594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:21.171321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:21.172950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:21.188631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:21.189993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:21.191473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:21.191473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:21.191473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:21.191473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 607/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:54:21.195806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:21.309672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e782-c963-4cba-9097-46920443d23a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:21.349393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:21.349995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:21.350278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:21.350551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:21.351273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:21.351583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:21.353127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:21.367408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 618/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:21.374271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:21.481138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:21.488534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bfeb8-75e1-49fd-a26e-05fc25a38c4d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:21.524213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:21.524857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:21.525136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:21.525936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:21.534311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:21.536003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:54:21.549628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:21.550894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:21.552043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 608/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:21.557371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:21.610785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:21.669970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822a4d4-c146-4240-902a-e616e57f4d4d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:21.708429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:21.709177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:21.709495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:21.709847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:21.710656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:21.710965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:21.712512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:21.738069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 619/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Oct 6 10:54:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:21.739679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:21.850868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382479e-2441-4c71-8068-919126fc6502 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:21.887910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:21.888497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:21.888780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:21.889054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:21.901520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:21.903419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:21.922325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:21.923484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:21.924496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:21.924496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:21.924496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:21.924496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 609/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:21.929982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:22.045239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b87d8-2cc3-4581-bbc8-324550f8463b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:22.047733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:22.084722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:22.085381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:22.085654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:22.086056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:22.086710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:22.087023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:22.089151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:22.100874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 620/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:22.107492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:22.116832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:22.218879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320baf8-e901-40d8-8fee-4152d4466f6a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:22.257198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:22.257872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:22.258153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:22.258439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:22.266275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:22.267800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:54:22.279550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:22.285769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:22.286689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 610/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:22.287278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:22.406890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a6504-bbfa-4ef2-932d-f01b99b21783 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:22.444761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0ba0d-72f3-4499-acc0-06c6052be73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:22.445433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ba0d-72f3-4499-acc0-06c6052be73a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:22.445714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ba0d-72f3-4499-acc0-06c6052be73a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:22.445985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ba0d-72f3-4499-acc0-06c6052be73a None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:22.446695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0ba0d-72f3-4499-acc0-06c6052be73a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:22.446976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0ba0d-72f3-4499-acc0-06c6052be73a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:22.449217 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0ba0d-72f3-4499-acc0-06c6052be73a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:22.462249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 621/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:54:22.468517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:22.538818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:22.580418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f2b41-94b3-4f61-9474-c72b3efb26e6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:22.616707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cca8f4-52f9-4ba9-87fb-1ebd17a2419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:22.617366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cca8f4-52f9-4ba9-87fb-1ebd17a2419d None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:22.617650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cca8f4-52f9-4ba9-87fb-1ebd17a2419d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:22.617928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cca8f4-52f9-4ba9-87fb-1ebd17a2419d None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/49cbfb2f4ddb42c0b299311615c7b0dc/roles/7565190481584409b1756297cec264cd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:22.633502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cca8f4-52f9-4ba9-87fb-1ebd17a2419d None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=49cbfb2f4ddb42c0b299311615c7b0dc, role_id=7565190481584409b1756297cec264cd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:22.635463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cca8f4-52f9-4ba9-87fb-1ebd17a2419d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:22.648319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 611/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:54:22 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/49cbfb2f4ddb42c0b299311615c7b0dc/roles/7565190481584409b1756297cec264cd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:54:24.893800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb18d-34fd-4136-a398-3c0865e69030 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:24.894456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb18d-34fd-4136-a398-3c0865e69030 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:24.894456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb18d-34fd-4136-a398-3c0865e69030 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:24.895957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 622/1233] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:24.902024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a335fe-59c1-439c-8b4f-91ba11530e1e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:24.902769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a335fe-59c1-439c-8b4f-91ba11530e1e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:24.902869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a335fe-59c1-439c-8b4f-91ba11530e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:24.925895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-62a335fe-59c1-439c-8b4f-91ba11530e1e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:24.927056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62a335fe-59c1-439c-8b4f-91ba11530e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:24.998720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 612/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:25.290770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb18d-34fd-4136-a398-3c0865e69030 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:25.322428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:25.411817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb18d-34fd-4136-a398-3c0865e69030 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:25.449653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:25.450351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:25.450650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:25.450957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:25.461750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:54:25.467512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:25.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:25.469775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 623/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Oct 6 10:54:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:25.475202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a335fe-59c1-439c-8b4f-91ba11530e1e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:25.528400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:25.584806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a335fe-59c1-439c-8b4f-91ba11530e1e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:25.622537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:25.623149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:25.623467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:25.623744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:25.624464 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:25.624792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:25.626344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:25.634831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 613/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Oct 6 10:54:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:25.640626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:25.732719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:25.750582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1fd4-0950-483e-93cc-b62f374c50dc None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:25.785861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:25.786506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:25.786765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:25.787036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:25.797577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:25.799023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:25.813801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:25.814958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:25.816072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:25.816072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:25.816072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:25.816072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 624/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:25.820750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:25.930602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aec0fc-4021-4104-b6ef-99e7f2b5e0c2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:25.939927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:25.968457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:25.969067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:25.969346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:25.969740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:25.970433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:25.970675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:25.972057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:25.984584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 614/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:25.990942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:26.108639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ef33a-8b11-4bb9-9fc4-88f46cb1c69a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:26.145262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:26.145960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:26.146438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:26.146565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:26.149723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:26.157065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:26.158553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:54:26.170896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:26.172146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:26.173343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 625/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:26.178127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:26.285612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122ee1a-0877-48de-a427-68b0f1b992b6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:26.320474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:26.321080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:26.321349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:26.321622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:26.322909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:26.323198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:26.324657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:26.345500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 615/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Oct 6 10:54:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:26.351709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:26.460997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96d70c-e75f-46ef-884e-9f69f5972e34 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:26.496521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:26.497125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:26.497407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:26.497680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:26.561207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:26.568576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:26.570046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:26.678638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 626/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:54:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:54:26.685965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:26.715943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:26.794974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512b78-ab85-4630-bbe1-c035926e5876 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:26.831826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:26.832470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:26.832759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:26.833064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:26.844815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:26.846415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:26.862790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:26.864089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:26.865271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:26.865271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:26.865271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:26.865271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 616/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:26.870593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:26.971326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:26.978079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17102f-0e69-4401-a416-c245ea54097e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:27.013415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:27.014250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:27.014349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:27.014597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:27.015241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:27.015692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:27.017043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:27.029801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 627/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:54:27.036121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:27.122053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:27.144044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ef9e7-d142-42cf-899a-1eb6f75b6b97 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:27.178907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:27.179532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:27.179806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:27.180105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:27.187588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:27.189162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:54:27.202848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:27.204021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:27.204914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 617/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:27.210958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:27.327227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678b00e-49ed-4b37-b00b-36f4008b4211 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:27.366305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc52db-70c9-44df-a883-d284899fddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:27.367042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc52db-70c9-44df-a883-d284899fddec None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:27.367259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc52db-70c9-44df-a883-d284899fddec None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:27.367533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc52db-70c9-44df-a883-d284899fddec None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:27.368386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc52db-70c9-44df-a883-d284899fddec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:27.368731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc52db-70c9-44df-a883-d284899fddec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:27.370368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc52db-70c9-44df-a883-d284899fddec None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:27.386971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 628/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:27.394807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:27.524096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65494002-efda-42bf-982f-8263aa8c7aac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:27.564321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfce4c-0d04-4886-9833-deeee35374c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:27.564970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfce4c-0d04-4886-9833-deeee35374c8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:27.565284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfce4c-0d04-4886-9833-deeee35374c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:27.565627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfce4c-0d04-4886-9833-deeee35374c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:27.566439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfce4c-0d04-4886-9833-deeee35374c8 None admin] RBAC: Adding query filter params (role.id=7565190481584409b1756297cec264cd, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=49cbfb2f4ddb42c0b299311615c7b0dc) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:27.566773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfce4c-0d04-4886-9833-deeee35374c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:27.568464 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfce4c-0d04-4886-9833-deeee35374c8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:27.575532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 618/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:54:27 2022] GET /identity/v3/role_assignments?user.id=49cbfb2f4ddb42c0b299311615c7b0dc&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=7565190481584409b1756297cec264cd => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:29.956250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2929d-349e-4737-b695-af546dbfcaa0 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:29.956807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2929d-349e-4737-b695-af546dbfcaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:29.956807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2929d-349e-4737-b695-af546dbfcaa0 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:29.958596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 629/1247] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:29.964727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3848-4896-4014-bf07-c7b2ba72acb6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:29.965138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3848-4896-4014-bf07-c7b2ba72acb6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:29.965276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3848-4896-4014-bf07-c7b2ba72acb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:29.972037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:29.985977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3f0b3848-4896-4014-bf07-c7b2ba72acb6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:29.988023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3f0b3848-4896-4014-bf07-c7b2ba72acb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:30.065854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 619/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:30.340713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2929d-349e-4737-b695-af546dbfcaa0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:30.359296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:30.453023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2929d-349e-4737-b695-af546dbfcaa0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:30.490144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:30.490863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:30.491193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:30.491517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:30.503061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:54:30.507920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:30.509124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:30.510768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 630/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:54:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:30.516439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3848-4896-4014-bf07-c7b2ba72acb6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:30.624282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3848-4896-4014-bf07-c7b2ba72acb6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:30.657891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:30.663651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:30.664371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:30.664679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:30.664951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:30.665620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:30.666067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:30.668493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:30.687654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 620/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:54:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:30.688928 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:30.774887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:30.802715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a9fef-1e0a-466d-b44f-2609adc3e269 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:30.840577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:30.841196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:30.841479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:30.841756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:30.854062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:30.855698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:30.872477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:30.873827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:30.874779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:30.874779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:30.874779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:30.874779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 631/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:30.879695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:30.989491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f510ad2-8e03-42cb-b9de-4a5ee05d41b6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:31.028967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:31.029686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:31.029964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:31.030278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:31.031075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:31.031378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:31.033091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:31.050195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 621/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:31.056326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:31.165532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c013db-4eeb-4ee9-b68a-44ba7cb4b57a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:31.194072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:31.203256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:31.203929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:31.204212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:31.204486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:31.213033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:31.214519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:54:31.226755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:31.228002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:31.229168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 632/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:31.234213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:31.292045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:31.337434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c7ba-95fe-4ef7-9528-521d6f61d63c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:31.372190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:31.372800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:31.373075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:31.373341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:31.374060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:31.374365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:31.375774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:31.393971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 622/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Oct 6 10:54:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:31.401392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:31.509175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae930aa-5019-413b-90ac-bd08a5c245a5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:31.546552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:31.547155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:31.547433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:31.547705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:31.602646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:31.614909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:31.616415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:31.712125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 633/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:54:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:54:31.718601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:31.814993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2889f8-742f-41f7-987e-f210d4712f4c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:31.828122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:31.848803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:31.849425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:31.849705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:31.849979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:31.859743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:31.861217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:31.875880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:31.876824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:31.877765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:31.877765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:31.877765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:31.877765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 623/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:31.882594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:31.967197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:31.979327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6b41f-e06a-44fc-9483-634313da0faa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:32.013371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:32.014138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:32.014420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:32.014759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:32.015447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:32.015739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:32.017607 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:32.030305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 634/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:32.036998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:32.135505 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc3555-97dc-4769-aa78-26678911c61b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:32.167180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:32.167793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:32.168102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:32.168373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:32.174669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:32.176115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:54:32.188386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:32.189433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:32.190590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 624/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:54:32.195426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:32.292513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cefac4-318d-442d-99c6-dfcdec61bc8d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:32.324768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58d845-959b-4a5a-bd28-e8cc1f2eef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:32.325348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58d845-959b-4a5a-bd28-e8cc1f2eef2f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:32.325623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58d845-959b-4a5a-bd28-e8cc1f2eef2f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:32.325893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58d845-959b-4a5a-bd28-e8cc1f2eef2f None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:32.326560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58d845-959b-4a5a-bd28-e8cc1f2eef2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:32.326862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58d845-959b-4a5a-bd28-e8cc1f2eef2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:32.328314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58d845-959b-4a5a-bd28-e8cc1f2eef2f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:32.339796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 635/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:32.345849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:32.371616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:32.444101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fffcf-7184-437f-ba35-9ddbae8c7cf1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:32.475586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21fcdf-ca51-4f99-8a46-3376664ec904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:32.476206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21fcdf-ca51-4f99-8a46-3376664ec904 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:32.476484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21fcdf-ca51-4f99-8a46-3376664ec904 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:32.476754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21fcdf-ca51-4f99-8a46-3376664ec904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:32.477474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21fcdf-ca51-4f99-8a46-3376664ec904 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=49cbfb2f4ddb42c0b299311615c7b0dc) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:32.477780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21fcdf-ca51-4f99-8a46-3376664ec904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:32.479823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21fcdf-ca51-4f99-8a46-3376664ec904 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:32.485012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 625/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:54:32 2022] GET /identity/v3/role_assignments?user.id=49cbfb2f4ddb42c0b299311615c7b0dc&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=4c41d809cc274ff5871dc8092f05715a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:34.650670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934901-d2cd-4b39-ab3f-c0cad9927bdd None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:34.651405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934901-d2cd-4b39-ab3f-c0cad9927bdd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:34.651685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934901-d2cd-4b39-ab3f-c0cad9927bdd None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:34.653334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 636/1261] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:34.658430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364033-4fed-46ce-ad9f-e45142ddddef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:34.658862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364033-4fed-46ce-ad9f-e45142ddddef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:34.659001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b364033-4fed-46ce-ad9f-e45142ddddef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:34.677105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7b364033-4fed-46ce-ad9f-e45142ddddef None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:34.678265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b364033-4fed-46ce-ad9f-e45142ddddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:34.742160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 626/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:35.011918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934901-d2cd-4b39-ab3f-c0cad9927bdd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:35.023509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:35.118450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934901-d2cd-4b39-ab3f-c0cad9927bdd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:35.151394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:35.152030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:35.152310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:35.152845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:35.162422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:54:35.167147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:35.168286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:35.169321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 637/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:54:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:35.174484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b364033-4fed-46ce-ad9f-e45142ddddef None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:35.192700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:35.270948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b364033-4fed-46ce-ad9f-e45142ddddef None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:35.304141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:35.304732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:35.305012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:35.305284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:35.305944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:35.306246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:35.308327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:35.316099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 627/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:54:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:35.321787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:35.389094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:35.423305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0be753-0951-4095-aea2-9d40c79bae67 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:35.457012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:35.457621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:35.457899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:35.458171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:35.468163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:35.469618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:35.484373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:35.485449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:35.486525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:35.486525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:35.486525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:35.486525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 638/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:35.491278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:35.571409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:35.591203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714f05d-a988-49ee-8949-f534c157cc86 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:35.624062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:35.624658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:35.624939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:35.625211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:35.625845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:35.626151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:35.627502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:35.638982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 628/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:35.644602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:35.743366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164be65-c01c-45e7-be18-2a66a1788cef None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:35.762176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:35.777226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:35.778098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:35.778376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:35.778646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:35.785782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:35.787210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:54:35.798582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:35.799702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:35.800716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 639/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:35.805517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:35.899648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf7165-7930-4fb8-8eb5-342fdebf90f2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:35.932798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:35.933386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:35.933664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:35.933937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:35.934650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:35.934951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:35.936390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:35.944631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:35.953855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 629/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Oct 6 10:54:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:35.959711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:36.061130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b93-9a79-4e65-8eda-38a239496a4a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:36.094886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:36.095510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:36.095792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:36.096094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:36.106624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:36.108101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:36.121979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:36.123090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:36.124218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:36.124218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:36.124218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:36.124218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 640/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:36.128783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:36.225643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5326-c16f-4e86-a864-7e6cffac6da0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:36.257796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:36.258390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:36.258672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:36.258947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:36.260204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:36.260644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:36.261995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:36.272799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 630/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:36.279429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:36.309758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:36.377750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af07afb-30e2-467e-991a-d8dbf7381f26 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:36.411451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:36.412111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:36.412392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:36.412660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:36.419157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:36.420631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:54:36.432661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:36.433729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:36.434851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 641/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:36.439498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:36.496433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:36.536864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53015d-74ac-4f99-a6d7-ccb431f62ee7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:36.569257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15849e75-fca2-4540-8cf7-59588c17575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:36.569851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15849e75-fca2-4540-8cf7-59588c17575b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:36.570132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15849e75-fca2-4540-8cf7-59588c17575b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:36.570405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15849e75-fca2-4540-8cf7-59588c17575b None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:36.571083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15849e75-fca2-4540-8cf7-59588c17575b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:36.571387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15849e75-fca2-4540-8cf7-59588c17575b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:36.573576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15849e75-fca2-4540-8cf7-59588c17575b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:36.585048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 631/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:36.591012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:36.679822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:36.689451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913c3b1-fab2-49cd-acf8-19331feb1949 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:36.721973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fc4d5-f036-4af3-8819-49aa9b5818fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:36.722607 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fc4d5-f036-4af3-8819-49aa9b5818fa None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:36.722886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fc4d5-f036-4af3-8819-49aa9b5818fa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:36.723160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fc4d5-f036-4af3-8819-49aa9b5818fa None admin] PATH_INFO: `/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/49cbfb2f4ddb42c0b299311615c7b0dc/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:36.725013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fc4d5-f036-4af3-8819-49aa9b5818fa None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0751f4839c4258b9a815a044d9c388, user_id=49cbfb2f4ddb42c0b299311615c7b0dc, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:36.726433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fc4d5-f036-4af3-8819-49aa9b5818fa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:36.739612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 642/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Oct 6 10:54:36 2022] PUT /identity/v3/projects/1a0751f4839c4258b9a815a044d9c388/users/49cbfb2f4ddb42c0b299311615c7b0dc/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 10:54:38.833376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de0d30-35c0-4d5f-a434-2e7ee4b05e76 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:38.833919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de0d30-35c0-4d5f-a434-2e7ee4b05e76 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:38.833919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de0d30-35c0-4d5f-a434-2e7ee4b05e76 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:38.835254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 632/1274] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:38.840636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bef4b-f292-43ba-895a-3e7df4bacb85 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:38.841055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bef4b-f292-43ba-895a-3e7df4bacb85 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:38.841176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bef4b-f292-43ba-895a-3e7df4bacb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:38.859094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-249bef4b-f292-43ba-895a-3e7df4bacb85 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:38.860293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-249bef4b-f292-43ba-895a-3e7df4bacb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:38.924896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 643/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:39.210863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de0d30-35c0-4d5f-a434-2e7ee4b05e76 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:39.309940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de0d30-35c0-4d5f-a434-2e7ee4b05e76 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:39.326504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:39.343907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:39.344527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:39.344805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:39.345079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:39.355759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 10:54:39.360529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:39.361642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:39.362906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 633/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Oct 6 10:54:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:39.367454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bef4b-f292-43ba-895a-3e7df4bacb85 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:39.447296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:39.466137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bef4b-f292-43ba-895a-3e7df4bacb85 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:39.498807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:39.499401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:39.499681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:39.499983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:39.500642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:39.500945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:39.502310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:39.510215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 644/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Oct 6 10:54:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:39.516263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:39.614683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744e9e2-cbd2-46fd-8d52-57e0af74685e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:39.647353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:39.647995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:39.648281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:39.648556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:39.659088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:39.660559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:39.675342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:39.676494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:39.677583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:39.677583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:39.677583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:39.677583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 634/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:39.682186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:39.779801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b627223-a208-4455-b84c-ad346c94ffa4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:39.812760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:39.813349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:39.813633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:39.813908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:39.814543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:39.814849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:39.816241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:39.828200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 645/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:39.833771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:39.864801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:39.932251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb595efa-291d-4394-aaa6-03bf853ea736 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:39.964628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:39.965240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:39.965516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:39.965785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:39.972867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:39.974276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 10:54:39.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:39.986113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:39.987609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 635/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:39.991764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:39.995584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:40.091504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad39ff-e972-402c-b2b2-a53497c2ef53 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:40.124073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:40.124667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:40.124948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:40.125221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:40.125933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:40.126238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:40.127680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:40.145407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 646/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Oct 6 10:54:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:40.151169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:40.238288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:40.247917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143bf2a-8401-473c-aea1-50bff1a0e35e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:40.280378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:40.280968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:40.281249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:40.281522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:40.345383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:40.346746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:40.432008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:40.442001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 636/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Oct 6 10:54:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 10:54:40.448558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:40.507785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:40.545603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c47f3-920a-45fb-9c70-46e0da2f58cf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:40.578382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:40.578989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:40.579269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:40.579543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:40.589195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:40.590619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 10:54:40.605307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:40.606442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 10:54:40.607359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 10:54:40.607359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 10:54:40.607359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:40.607359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 647/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 10:54:40.612160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:40.708725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016843-8836-4a8a-ba1d-0acd81246ca9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:40.741281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:40.741874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:40.742156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:40.742432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:40.743068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:40.743375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:40.745368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:40.756132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 637/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Oct 6 10:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:40.761730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:40.859328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e3d0-0b50-4426-b74c-3a037bb643fd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:40.872908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:40.892916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:40.893523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:40.893800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:40.894071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:40.901031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:40.902745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 10:54:40.913600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:40.914620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Oct 06 10:54:40.915575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 648/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Oct 6 10:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:40.921054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:40.972750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:41.023726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b9dea-292c-42b6-9979-2ddc2d2b27b0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:41.058223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f057b2c-e258-4b52-818d-ec8e3831d5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:41.058821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f057b2c-e258-4b52-818d-ec8e3831d5fc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:41.059101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f057b2c-e258-4b52-818d-ec8e3831d5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:41.059373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f057b2c-e258-4b52-818d-ec8e3831d5fc None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:41.060086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f057b2c-e258-4b52-818d-ec8e3831d5fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:41.060391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f057b2c-e258-4b52-818d-ec8e3831d5fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:41.061801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f057b2c-e258-4b52-818d-ec8e3831d5fc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:41.074170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 638/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Oct 6 10:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 10:54:41.080233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:41.177139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb71cf3-4fc0-4556-98bd-a670c94935af None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:41.209251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2a1e8-152d-40d5-b467-48e361d6c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:41.209833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2a1e8-152d-40d5-b467-48e361d6c504 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:41.210110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2a1e8-152d-40d5-b467-48e361d6c504 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:41.210380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2a1e8-152d-40d5-b467-48e361d6c504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:41.211096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2a1e8-152d-40d5-b467-48e361d6c504 None admin] RBAC: Adding query filter params (role.id=4c41d809cc274ff5871dc8092f05715a, scope.project.id=1a0751f4839c4258b9a815a044d9c388, user.id=49cbfb2f4ddb42c0b299311615c7b0dc) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:41.211401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2a1e8-152d-40d5-b467-48e361d6c504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:41.213366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2a1e8-152d-40d5-b467-48e361d6c504 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:41.218855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 649/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Oct 6 10:54:41 2022] GET /identity/v3/role_assignments?user.id=49cbfb2f4ddb42c0b299311615c7b0dc&scope.project.id=1a0751f4839c4258b9a815a044d9c388&role.id=4c41d809cc274ff5871dc8092f05715a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:43.412472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0888807-ec15-4641-8d14-37aeac73c1a7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:43.412928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0888807-ec15-4641-8d14-37aeac73c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:43.413023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0888807-ec15-4641-8d14-37aeac73c1a7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:43.414350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 639/1288] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:43.419703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1f9d-1599-4c60-9031-be6567acc3d6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:43.420155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1f9d-1599-4c60-9031-be6567acc3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:43.420295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1f9d-1599-4c60-9031-be6567acc3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:43.438593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-51ac1f9d-1599-4c60-9031-be6567acc3d6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:43.439773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-51ac1f9d-1599-4c60-9031-be6567acc3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:43.468818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:43.505834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 650/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:43.775654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0888807-ec15-4641-8d14-37aeac73c1a7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:43.874575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0888807-ec15-4641-8d14-37aeac73c1a7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:43.884937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:43.908592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:43.909210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:43.909491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:43.909764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:43.910342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:43.911691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:43.920948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:43.922164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:43.923193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 640/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:54:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:43.928108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac1f9d-1599-4c60-9031-be6567acc3d6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:44.025499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac1f9d-1599-4c60-9031-be6567acc3d6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:44.066847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc4309-be6b-42af-a6c4-324f50d99374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:44.067500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4309-be6b-42af-a6c4-324f50d99374 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:44.067807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4309-be6b-42af-a6c4-324f50d99374 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:44.068160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4309-be6b-42af-a6c4-324f50d99374 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:44.068848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc4309-be6b-42af-a6c4-324f50d99374 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:44.069175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc4309-be6b-42af-a6c4-324f50d99374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:44.070765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc4309-be6b-42af-a6c4-324f50d99374 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:44.076319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 651/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Oct 6 10:54:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:44.083069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:44.180536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cc064-d2f5-4cc2-bc4c-ed909da923e3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:44.212091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab7d0e-6c4e-4f37-bbad-7c632667eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:44.213190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab7d0e-6c4e-4f37-bbad-7c632667eb93 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:44.213462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab7d0e-6c4e-4f37-bbad-7c632667eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:44.213731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab7d0e-6c4e-4f37-bbad-7c632667eb93 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:44.214344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab7d0e-6c4e-4f37-bbad-7c632667eb93 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:44.214641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab7d0e-6c4e-4f37-bbad-7c632667eb93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:44.216003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab7d0e-6c4e-4f37-bbad-7c632667eb93 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:44.220250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 641/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Oct 6 10:54:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:46.346327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978fa5-15fd-4dfb-a8f4-84773e1a4bec None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:46.346896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978fa5-15fd-4dfb-a8f4-84773e1a4bec None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:46.346896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978fa5-15fd-4dfb-a8f4-84773e1a4bec None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:46.348254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 652/1293] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:46.353439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd6c6-5d96-4e5f-bde0-6d8f2dd5c8e1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:46.353838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd6c6-5d96-4e5f-bde0-6d8f2dd5c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:46.353993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd6c6-5d96-4e5f-bde0-6d8f2dd5c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:46.372308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2e0dd6c6-5d96-4e5f-bde0-6d8f2dd5c8e1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:46.373492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2e0dd6c6-5d96-4e5f-bde0-6d8f2dd5c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:46.405704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:46.442032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 642/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:46.708261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978fa5-15fd-4dfb-a8f4-84773e1a4bec None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:46.739363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:46.809726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978fa5-15fd-4dfb-a8f4-84773e1a4bec None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:46.842665 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c763014-1648-4407-97d1-8731f55e2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:46.843252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763014-1648-4407-97d1-8731f55e2d80 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:46.843532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763014-1648-4407-97d1-8731f55e2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:46.843804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763014-1648-4407-97d1-8731f55e2d80 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:46.844480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c763014-1648-4407-97d1-8731f55e2d80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:46.845845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c763014-1648-4407-97d1-8731f55e2d80 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:46.860155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 653/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Oct 6 10:54:46 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:54:49.027587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bd00b-536e-4ef1-bc68-07d5d9e9e6ed None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:49.028171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bd00b-536e-4ef1-bc68-07d5d9e9e6ed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:49.028357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bd00b-536e-4ef1-bc68-07d5d9e9e6ed None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:49.029861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 643/1296] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:49.036558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f00a7f-c6d1-4f76-b574-504a033c7fe7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:49.036954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f00a7f-c6d1-4f76-b574-504a033c7fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:49.037156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f00a7f-c6d1-4f76-b574-504a033c7fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:49.059757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-49f00a7f-c6d1-4f76-b574-504a033c7fe7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:49.060979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-49f00a7f-c6d1-4f76-b574-504a033c7fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:49.139479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 654/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:49.411599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bd00b-536e-4ef1-bc68-07d5d9e9e6ed None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:49.510270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bd00b-536e-4ef1-bc68-07d5d9e9e6ed None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:49.543259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:49.543892 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:49.544173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:49.544447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:49.545022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:49.546371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:49.555000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:49.556169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:49.557028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 644/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:54:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:49.561953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f00a7f-c6d1-4f76-b574-504a033c7fe7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:49.580327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:49.659430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f00a7f-c6d1-4f76-b574-504a033c7fe7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:49.691337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:49.691949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:49.692231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:49.692502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:49.693123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:49.693421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:49.694773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:49.700256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 655/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Oct 6 10:54:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:49.706125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:49.737193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:49.803707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed64fa-9507-42b9-a148-5b239af6813b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:49.836057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbfffe-7c87-421f-86ca-8ef15fe521f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:49.836649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbfffe-7c87-421f-86ca-8ef15fe521f9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:49.836931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbfffe-7c87-421f-86ca-8ef15fe521f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:49.837206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbfffe-7c87-421f-86ca-8ef15fe521f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:49.838523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbfffe-7c87-421f-86ca-8ef15fe521f9 None admin] RBAC: Adding query filter params (interface=public, service_id=223faecfa0124840bb944b437549470c, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:49.838834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbfffe-7c87-421f-86ca-8ef15fe521f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:49.840282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbfffe-7c87-421f-86ca-8ef15fe521f9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:49.845120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 645/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Oct 6 10:54:49 2022] GET /identity/v3/endpoints?service_id=223faecfa0124840bb944b437549470c&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:49.850577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:49.932484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:49.951094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0040737-3788-420a-955e-d28f4809a7f4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:49.984929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d7ff4-df26-4061-80cc-5c4f9ed42021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:49.985521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d7ff4-df26-4061-80cc-5c4f9ed42021 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:49.985802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d7ff4-df26-4061-80cc-5c4f9ed42021 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:49.986076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d7ff4-df26-4061-80cc-5c4f9ed42021 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:49.986669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d7ff4-df26-4061-80cc-5c4f9ed42021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:49.988062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d7ff4-df26-4061-80cc-5c4f9ed42021 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:49.994873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 656/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Oct 6 10:54:49 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 10:54:52.100200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c76c-88b3-44ec-a08e-d1109e09b3a3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:52.100764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c76c-88b3-44ec-a08e-d1109e09b3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:52.100764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c76c-88b3-44ec-a08e-d1109e09b3a3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:52.102077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 646/1302] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:54:52.107201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a58e0-585d-4764-83c3-130b1653af91 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:52.107644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a58e0-585d-4764-83c3-130b1653af91 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:52.107748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a58e0-585d-4764-83c3-130b1653af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:52.126021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-189a58e0-585d-4764-83c3-130b1653af91 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:54:52.127198 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-189a58e0-585d-4764-83c3-130b1653af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:54:52.192330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 657/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:54:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 10:54:52.460315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c76c-88b3-44ec-a08e-d1109e09b3a3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:52.552647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:52.560460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c76c-88b3-44ec-a08e-d1109e09b3a3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:52.592363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:52.592972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:52.593250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:52.593522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:52.594090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:52.595435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 10:54:52.604477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 10:54:52.605687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Oct 06 10:54:52.606552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 647/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Oct 6 10:54:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 10:54:52.611678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a58e0-585d-4764-83c3-130b1653af91 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:52.689672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:54:52.712292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a58e0-585d-4764-83c3-130b1653af91 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:52.746646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720d0e9-43b7-4a3e-8960-ae5f2387311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:52.747242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720d0e9-43b7-4a3e-8960-ae5f2387311a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:52.747524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720d0e9-43b7-4a3e-8960-ae5f2387311a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:52.747796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720d0e9-43b7-4a3e-8960-ae5f2387311a None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:52.748450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720d0e9-43b7-4a3e-8960-ae5f2387311a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 10:54:52.748750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720d0e9-43b7-4a3e-8960-ae5f2387311a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:52.750107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720d0e9-43b7-4a3e-8960-ae5f2387311a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:52.755084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 658/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Oct 6 10:54:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 10:54:52.760889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:54:52.857178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022255c2-106b-4a3f-a17d-d3a1024caebd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:54:52.889009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58871cad-6d77-4d80-aed2-3a007bc68a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:54:52.889595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58871cad-6d77-4d80-aed2-3a007bc68a0a None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:54:52.889874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58871cad-6d77-4d80-aed2-3a007bc68a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:54:52.890143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58871cad-6d77-4d80-aed2-3a007bc68a0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:54:52.890802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58871cad-6d77-4d80-aed2-3a007bc68a0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:54:52.892179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58871cad-6d77-4d80-aed2-3a007bc68a0a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:54:52.911271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 648/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 10:54:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 10:55:02.752242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=30 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:55:02.907403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:55:56.778443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31043e-54d8-4593-a2a0-fa98aae14126 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:55:56.778895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31043e-54d8-4593-a2a0-fa98aae14126 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:55:56.778979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31043e-54d8-4593-a2a0-fa98aae14126 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:55:56.780334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 659/1307] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:55:56.785710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51997cc-592b-438b-9cf8-eab981db5370 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:55:56.786133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51997cc-592b-438b-9cf8-eab981db5370 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:55:56.786296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51997cc-592b-438b-9cf8-eab981db5370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:55:56.953392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f51997cc-592b-438b-9cf8-eab981db5370 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:55:56.954604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f51997cc-592b-438b-9cf8-eab981db5370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:55:57.112595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 649/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:55:57.404031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e61567-e8d7-49e2-b090-279359d1a111 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:55:57.404312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e61567-e8d7-49e2-b090-279359d1a111 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:55:57.404590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e61567-e8d7-49e2-b090-279359d1a111 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:55:57.406132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 660/1309] 10.222.0.52 () {54 vars in 964 bytes} [Thu Oct 6 10:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:55:57.413226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8666-6e47-49ae-b04f-df06927c05a3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:55:57.413554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8666-6e47-49ae-b04f-df06927c05a3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:55:57.413925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8666-6e47-49ae-b04f-df06927c05a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:55:57.514434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a1cc8666-6e47-49ae-b04f-df06927c05a3 None None] MFA Rules not processed for user `b5c892640b3e409b9a99b4ca1d34a964`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:55:57.515695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a1cc8666-6e47-49ae-b04f-df06927c05a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:55:57.541195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:55:57.608417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 650/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Oct 6 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:55:57.617807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e61567-e8d7-49e2-b090-279359d1a111 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:55:57.619684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-04e61567-e8d7-49e2-b090-279359d1a111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:55:57.741316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e61567-e8d7-49e2-b090-279359d1a111 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:55:57.825609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7a1ce-45b4-4e59-9454-e6205bb9f92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:55:57.827147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7a1ce-45b4-4e59-9454-e6205bb9f92f service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:55:57.827768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7a1ce-45b4-4e59-9454-e6205bb9f92f service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:55:57.828274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7a1ce-45b4-4e59-9454-e6205bb9f92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:55:57.830090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-97a7a1ce-45b4-4e59-9454-e6205bb9f92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:55:57.891026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:55:57.953632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7a1ce-45b4-4e59-9454-e6205bb9f92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:55:57.955555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7a1ce-45b4-4e59-9454-e6205bb9f92f service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:55:57.976271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 661/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:00.462975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c2b61-fe8d-4adf-94e7-e123f8049c4a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:00.463475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c2b61-fe8d-4adf-94e7-e123f8049c4a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:00.463475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c2b61-fe8d-4adf-94e7-e123f8049c4a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:00.464833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 651/1312] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:00.470167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1ace-3fda-4a34-8a25-e198c100346b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:00.470580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1ace-3fda-4a34-8a25-e198c100346b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:00.470719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1ace-3fda-4a34-8a25-e198c100346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:00.542070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-26fa1ace-3fda-4a34-8a25-e198c100346b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:00.543265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-26fa1ace-3fda-4a34-8a25-e198c100346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:00.639601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 662/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:00.926333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c2b61-fe8d-4adf-94e7-e123f8049c4a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:00.927887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-833c2b61-fe8d-4adf-94e7-e123f8049c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:01.032927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c2b61-fe8d-4adf-94e7-e123f8049c4a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:01.103140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:01.105809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e541e5-8afd-4902-9837-8e30e6649f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:01.106646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e541e5-8afd-4902-9837-8e30e6649f17 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:01.107120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e541e5-8afd-4902-9837-8e30e6649f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:01.107442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e541e5-8afd-4902-9837-8e30e6649f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:01.109249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-88e541e5-8afd-4902-9837-8e30e6649f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:01.215919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e541e5-8afd-4902-9837-8e30e6649f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:01.217284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e541e5-8afd-4902-9837-8e30e6649f17 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:01.235981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 652/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:03.681736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298dfad-4b23-4465-9b14-186abeabfd12 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:03.682265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298dfad-4b23-4465-9b14-186abeabfd12 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:03.682265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298dfad-4b23-4465-9b14-186abeabfd12 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:03.683578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 663/1315] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:03.689057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6d655-9f20-4741-aa23-f6491a4e645d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:03.689464 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6d655-9f20-4741-aa23-f6491a4e645d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:03.689723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6d655-9f20-4741-aa23-f6491a4e645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:03.756512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8fb6d655-9f20-4741-aa23-f6491a4e645d None None] MFA Rules not processed for user `7db7540927364ffabd7480f25538841b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:03.757724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8fb6d655-9f20-4741-aa23-f6491a4e645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:03.809041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:03.853713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 653/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:04.139700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298dfad-4b23-4465-9b14-186abeabfd12 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:04.141263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e298dfad-4b23-4465-9b14-186abeabfd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:04.151071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:04.243352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298dfad-4b23-4465-9b14-186abeabfd12 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:04.315449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007136f-cea6-44a0-9e29-3a78811af347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:04.316072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007136f-cea6-44a0-9e29-3a78811af347 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:04.316352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007136f-cea6-44a0-9e29-3a78811af347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:04.316622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007136f-cea6-44a0-9e29-3a78811af347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:04.318199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e007136f-cea6-44a0-9e29-3a78811af347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:04.378805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:04.427205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007136f-cea6-44a0-9e29-3a78811af347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:04.428680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007136f-cea6-44a0-9e29-3a78811af347 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:04.447946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 664/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:07.192443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221dba4-6b9a-4311-ae60-b6df378c106d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:07.193020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221dba4-6b9a-4311-ae60-b6df378c106d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:07.193020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221dba4-6b9a-4311-ae60-b6df378c106d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:07.194338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 654/1318] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:07.203040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008b0fb-045a-45ff-8f11-b8684bd13b1c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:07.203505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008b0fb-045a-45ff-8f11-b8684bd13b1c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:07.203600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008b0fb-045a-45ff-8f11-b8684bd13b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:07.271254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9008b0fb-045a-45ff-8f11-b8684bd13b1c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:07.272512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9008b0fb-045a-45ff-8f11-b8684bd13b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:07.384773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 665/1319] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:07.673710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221dba4-6b9a-4311-ae60-b6df378c106d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:07.675779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8221dba4-6b9a-4311-ae60-b6df378c106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:07.791982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221dba4-6b9a-4311-ae60-b6df378c106d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:07.864507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa3b92-f4b1-4581-8688-2a5e818ac45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:07.865115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa3b92-f4b1-4581-8688-2a5e818ac45b service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:07.865397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa3b92-f4b1-4581-8688-2a5e818ac45b service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:07.865668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa3b92-f4b1-4581-8688-2a5e818ac45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:07.867754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d9fa3b92-f4b1-4581-8688-2a5e818ac45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:07.875016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:07.976509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa3b92-f4b1-4581-8688-2a5e818ac45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:07.978125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa3b92-f4b1-4581-8688-2a5e818ac45b service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:07.996283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 655/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:10.798524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4b2d-6f35-482b-9b69-29727a31367e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:10.799086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4b2d-6f35-482b-9b69-29727a31367e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:10.799086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4b2d-6f35-482b-9b69-29727a31367e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:10.800574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 666/1321] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:10.806625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390509-c352-4d38-8d7f-8f86807de9a6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:10.807023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390509-c352-4d38-8d7f-8f86807de9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:10.807170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390509-c352-4d38-8d7f-8f86807de9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:10.879365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-87390509-c352-4d38-8d7f-8f86807de9a6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:10.880645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-87390509-c352-4d38-8d7f-8f86807de9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:10.943416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:10.987252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 656/1322] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:11.268308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4b2d-6f35-482b-9b69-29727a31367e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:11.269874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f04d4b2d-6f35-482b-9b69-29727a31367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:11.290466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:11.374868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4b2d-6f35-482b-9b69-29727a31367e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:11.450187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5d8bb-8eed-46ef-841b-3e99b05f7a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:11.450794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5d8bb-8eed-46ef-841b-3e99b05f7a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:11.451069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5d8bb-8eed-46ef-841b-3e99b05f7a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:11.451337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5d8bb-8eed-46ef-841b-3e99b05f7a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:11.452940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-00d5d8bb-8eed-46ef-841b-3e99b05f7a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:11.529245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:11.567369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5d8bb-8eed-46ef-841b-3e99b05f7a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:11.568839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5d8bb-8eed-46ef-841b-3e99b05f7a13 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:11.588369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 667/1323] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:13.807108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e285c-e099-4fa3-8d56-4ab498b1a30f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:13.807688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e285c-e099-4fa3-8d56-4ab498b1a30f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:13.807688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e285c-e099-4fa3-8d56-4ab498b1a30f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:13.809258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 657/1324] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:56:13.814450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e3c5f-9c09-4d2b-83b1-561d38f05e0a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:13.814880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e3c5f-9c09-4d2b-83b1-561d38f05e0a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:13.814999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e3c5f-9c09-4d2b-83b1-561d38f05e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:13.881470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3c6e3c5f-9c09-4d2b-83b1-561d38f05e0a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:13.882656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3c6e3c5f-9c09-4d2b-83b1-561d38f05e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:13.987646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 668/1325] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:14.276024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e285c-e099-4fa3-8d56-4ab498b1a30f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:14.277891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd5e285c-e099-4fa3-8d56-4ab498b1a30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:14.380913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e285c-e099-4fa3-8d56-4ab498b1a30f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:14.460187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f143585-40bc-4d63-a3ce-8d3398348363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:14.460800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143585-40bc-4d63-a3ce-8d3398348363 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:14.461138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143585-40bc-4d63-a3ce-8d3398348363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:14.461404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143585-40bc-4d63-a3ce-8d3398348363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:14.463011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5f143585-40bc-4d63-a3ce-8d3398348363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:14.480131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:14.573695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f143585-40bc-4d63-a3ce-8d3398348363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:14.575069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f143585-40bc-4d63-a3ce-8d3398348363 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:14.593632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 658/1326] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:16.809234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca259d8d-1bcc-4893-93c7-182034660235 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:16.809753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca259d8d-1bcc-4893-93c7-182034660235 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:16.809753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca259d8d-1bcc-4893-93c7-182034660235 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:16.811120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 669/1327] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:16.816546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c61eb-02e7-4071-a038-a3b29c7bd2c8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:16.816990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c61eb-02e7-4071-a038-a3b29c7bd2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:16.817096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c61eb-02e7-4071-a038-a3b29c7bd2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:16.885725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-131c61eb-02e7-4071-a038-a3b29c7bd2c8 None None] MFA Rules not processed for user `7db7540927364ffabd7480f25538841b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:16.886860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-131c61eb-02e7-4071-a038-a3b29c7bd2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:16.951210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:16.987633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 659/1328] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:17.273520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca259d8d-1bcc-4893-93c7-182034660235 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:17.275092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ca259d8d-1bcc-4893-93c7-182034660235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:17.310377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:17.387424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca259d8d-1bcc-4893-93c7-182034660235 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:17.465588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb3e5-11a0-4ad7-b32b-d2d3b306de25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:17.466116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb3e5-11a0-4ad7-b32b-d2d3b306de25 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:17.466396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb3e5-11a0-4ad7-b32b-d2d3b306de25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:17.466673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb3e5-11a0-4ad7-b32b-d2d3b306de25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:17.468297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-88dcb3e5-11a0-4ad7-b32b-d2d3b306de25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:17.551624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:17.584252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcb3e5-11a0-4ad7-b32b-d2d3b306de25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:17.585708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcb3e5-11a0-4ad7-b32b-d2d3b306de25 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:17.604558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 670/1329] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:19.996849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a16138-c82e-42ba-8425-65437b1ca8d1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:19.997398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a16138-c82e-42ba-8425-65437b1ca8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:19.997398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a16138-c82e-42ba-8425-65437b1ca8d1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:19.998676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 660/1330] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:20.003944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc3de-504d-40f1-bf8d-af31cd6eaaa2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:20.004392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc3de-504d-40f1-bf8d-af31cd6eaaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:20.004491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc3de-504d-40f1-bf8d-af31cd6eaaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:20.071220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0a7dc3de-504d-40f1-bf8d-af31cd6eaaa2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:20.072442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0a7dc3de-504d-40f1-bf8d-af31cd6eaaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:20.168790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 671/1331] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:20.450996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a16138-c82e-42ba-8425-65437b1ca8d1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:20.452646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8a16138-c82e-42ba-8425-65437b1ca8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:20.557562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a16138-c82e-42ba-8425-65437b1ca8d1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:20.631165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1931d7-0d17-451f-93a8-664fea80ccca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:20.631766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1931d7-0d17-451f-93a8-664fea80ccca service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:20.632073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1931d7-0d17-451f-93a8-664fea80ccca service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:20.632346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1931d7-0d17-451f-93a8-664fea80ccca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:20.634565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ed1931d7-0d17-451f-93a8-664fea80ccca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:20.660640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:20.740781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1931d7-0d17-451f-93a8-664fea80ccca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:20.742766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1931d7-0d17-451f-93a8-664fea80ccca service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:20.761022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 661/1332] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:23.026575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bea46-fa04-421d-904f-5678e712f1c5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:23.027137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bea46-fa04-421d-904f-5678e712f1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:23.027137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bea46-fa04-421d-904f-5678e712f1c5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:23.028805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 672/1333] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:23.034183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0f888-ad75-47aa-b4e9-15e8e074083a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:23.034582 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0f888-ad75-47aa-b4e9-15e8e074083a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:23.034745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0f888-ad75-47aa-b4e9-15e8e074083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:23.099600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-acb0f888-ad75-47aa-b4e9-15e8e074083a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:23.100836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-acb0f888-ad75-47aa-b4e9-15e8e074083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:23.175198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:23.198148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 662/1334] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:23.484789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bea46-fa04-421d-904f-5678e712f1c5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:23.486304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6a8bea46-fa04-421d-904f-5678e712f1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:23.526289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:23.590801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bea46-fa04-421d-904f-5678e712f1c5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:23.663453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da093aad-8c39-4445-a644-97562a3be5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:23.664098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da093aad-8c39-4445-a644-97562a3be5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:23.664389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da093aad-8c39-4445-a644-97562a3be5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:23.664659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da093aad-8c39-4445-a644-97562a3be5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:23.666246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-da093aad-8c39-4445-a644-97562a3be5b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:23.749596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:23.775408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da093aad-8c39-4445-a644-97562a3be5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:23.776868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da093aad-8c39-4445-a644-97562a3be5b7 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:23.794859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 673/1335] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:26.256607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935e8a8-54a9-4c56-9f93-f6981ef35bcc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:26.257255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935e8a8-54a9-4c56-9f93-f6981ef35bcc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:26.257255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935e8a8-54a9-4c56-9f93-f6981ef35bcc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:26.259221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 663/1336] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:26.264428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360966bd-7e25-4c2b-8a2c-55eb3f31887e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:26.264856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360966bd-7e25-4c2b-8a2c-55eb3f31887e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:26.264985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360966bd-7e25-4c2b-8a2c-55eb3f31887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:26.330874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-360966bd-7e25-4c2b-8a2c-55eb3f31887e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:26.332110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-360966bd-7e25-4c2b-8a2c-55eb3f31887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:26.429473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 674/1337] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:26.723089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935e8a8-54a9-4c56-9f93-f6981ef35bcc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:26.724679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1935e8a8-54a9-4c56-9f93-f6981ef35bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:26.831800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935e8a8-54a9-4c56-9f93-f6981ef35bcc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:26.905340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a34eb5-9340-4ad1-9a07-73f706616ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:26.905949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a34eb5-9340-4ad1-9a07-73f706616ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:26.906229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a34eb5-9340-4ad1-9a07-73f706616ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:26.906501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a34eb5-9340-4ad1-9a07-73f706616ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:26.908119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c0a34eb5-9340-4ad1-9a07-73f706616ac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:26.943400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:27.019921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a34eb5-9340-4ad1-9a07-73f706616ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:27.021361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a34eb5-9340-4ad1-9a07-73f706616ac8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:27.039858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 664/1338] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:31.125119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc571be9-b0a3-4887-9027-bd685ba61aa3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:31.125628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc571be9-b0a3-4887-9027-bd685ba61aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:31.125628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc571be9-b0a3-4887-9027-bd685ba61aa3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:31.126931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 675/1339] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:31.132275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969514a-4531-410c-b96a-464939f883ba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:31.133453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969514a-4531-410c-b96a-464939f883ba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:31.133556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969514a-4531-410c-b96a-464939f883ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:31.201484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f969514a-4531-410c-b96a-464939f883ba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:31.202677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f969514a-4531-410c-b96a-464939f883ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:31.287330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:31.298998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 665/1340] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:31.583427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc571be9-b0a3-4887-9027-bd685ba61aa3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:31.585100 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fc571be9-b0a3-4887-9027-bd685ba61aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:31.635183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:31.691541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc571be9-b0a3-4887-9027-bd685ba61aa3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:31.765834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13269cfd-4fad-4406-8c72-f8bd95a86fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:31.766438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269cfd-4fad-4406-8c72-f8bd95a86fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:31.766719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269cfd-4fad-4406-8c72-f8bd95a86fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:31.766994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269cfd-4fad-4406-8c72-f8bd95a86fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:31.768623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-13269cfd-4fad-4406-8c72-f8bd95a86fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:31.861900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:31.879061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13269cfd-4fad-4406-8c72-f8bd95a86fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:31.880529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13269cfd-4fad-4406-8c72-f8bd95a86fc6 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:31.899227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 676/1341] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:34.745456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05808f5-faa7-40d2-af6a-13d606e13555 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:34.746026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05808f5-faa7-40d2-af6a-13d606e13555 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:34.746026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05808f5-faa7-40d2-af6a-13d606e13555 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:34.747338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 666/1342] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:34.752494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd86b8-b2bd-48ee-86ef-9ee51d9f81d9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:34.752921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd86b8-b2bd-48ee-86ef-9ee51d9f81d9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:34.753108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd86b8-b2bd-48ee-86ef-9ee51d9f81d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:34.818993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-75fd86b8-b2bd-48ee-86ef-9ee51d9f81d9 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:34.820763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-75fd86b8-b2bd-48ee-86ef-9ee51d9f81d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:34.924464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 677/1343] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:35.211051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05808f5-faa7-40d2-af6a-13d606e13555 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:35.212755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a05808f5-faa7-40d2-af6a-13d606e13555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:35.318018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05808f5-faa7-40d2-af6a-13d606e13555 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:35.392520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d4047-e26c-4404-b5ac-4e0279a8aa0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:35.393130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d4047-e26c-4404-b5ac-4e0279a8aa0d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:35.393402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d4047-e26c-4404-b5ac-4e0279a8aa0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:35.393669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d4047-e26c-4404-b5ac-4e0279a8aa0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:35.395284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-686d4047-e26c-4404-b5ac-4e0279a8aa0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:35.442903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:35.510789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d4047-e26c-4404-b5ac-4e0279a8aa0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:35.512247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d4047-e26c-4404-b5ac-4e0279a8aa0d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:35.530791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 667/1344] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:39.686216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f1bca-856e-4db8-b572-5b833bbaab39 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:39.686729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f1bca-856e-4db8-b572-5b833bbaab39 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:39.686856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f1bca-856e-4db8-b572-5b833bbaab39 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:39.688144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 678/1345] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:39.693417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bfd11-5563-4d43-a981-2ef98a7efca5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:39.693834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bfd11-5563-4d43-a981-2ef98a7efca5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:39.693968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bfd11-5563-4d43-a981-2ef98a7efca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:39.766491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-288bfd11-5563-4d43-a981-2ef98a7efca5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:39.767718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-288bfd11-5563-4d43-a981-2ef98a7efca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:39.864634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:39.867269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 668/1346] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:40.155466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f1bca-856e-4db8-b572-5b833bbaab39 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:40.157033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-887f1bca-856e-4db8-b572-5b833bbaab39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:40.220405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:40.269474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f1bca-856e-4db8-b572-5b833bbaab39 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:40.348515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed775d58-91e9-465c-96f3-86e5ceb0dd9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:40.349124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775d58-91e9-465c-96f3-86e5ceb0dd9a service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:40.349402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775d58-91e9-465c-96f3-86e5ceb0dd9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:40.349675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775d58-91e9-465c-96f3-86e5ceb0dd9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:40.352037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ed775d58-91e9-465c-96f3-86e5ceb0dd9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:40.466526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:40.471044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed775d58-91e9-465c-96f3-86e5ceb0dd9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:40.472974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed775d58-91e9-465c-96f3-86e5ceb0dd9a service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:40.494034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 679/1347] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:45.393905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3b08-bc72-4d8e-9c15-e33877619a9f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:45.394279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3b08-bc72-4d8e-9c15-e33877619a9f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:45.394279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3b08-bc72-4d8e-9c15-e33877619a9f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:45.401989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 669/1348] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 10:56:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:45.410270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87610623-8d71-4e3a-9be6-596933160f35 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:45.410270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87610623-8d71-4e3a-9be6-596933160f35 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:45.410270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87610623-8d71-4e3a-9be6-596933160f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:45.503212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-87610623-8d71-4e3a-9be6-596933160f35 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:45.510137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-87610623-8d71-4e3a-9be6-596933160f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:45.691979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 680/1349] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 10:56:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:49.953716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58483d-6a49-4bcb-8290-bda7f5d06853 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:49.954737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58483d-6a49-4bcb-8290-bda7f5d06853 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:49.955086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58483d-6a49-4bcb-8290-bda7f5d06853 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:49.956615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 670/1350] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 10:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:49.962304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a2dc5-4fb6-4e61-869a-ca90380ba9f7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:49.962757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a2dc5-4fb6-4e61-869a-ca90380ba9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:49.962842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a2dc5-4fb6-4e61-869a-ca90380ba9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:50.019862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:50.045649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8e1a2dc5-4fb6-4e61-869a-ca90380ba9f7 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:50.046863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8e1a2dc5-4fb6-4e61-869a-ca90380ba9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:50.168335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 681/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:51.392699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febaf9d7-e656-4039-948c-4fd31d91fd29 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:51.393153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febaf9d7-e656-4039-948c-4fd31d91fd29 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:51.393248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febaf9d7-e656-4039-948c-4fd31d91fd29 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:51.394577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 671/1352] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 10:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:51.401717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3bb9f-2ef3-4a59-b3d1-ea1a1ee988b9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:51.402222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3bb9f-2ef3-4a59-b3d1-ea1a1ee988b9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:51.402327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3bb9f-2ef3-4a59-b3d1-ea1a1ee988b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:51.478882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-81e3bb9f-2ef3-4a59-b3d1-ea1a1ee988b9 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:51.480102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-81e3bb9f-2ef3-4a59-b3d1-ea1a1ee988b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:51.521493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:51.587883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 682/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:55.183917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad4db-1ab5-4228-9b23-695f11a33bed None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:55.184435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad4db-1ab5-4228-9b23-695f11a33bed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:55.184435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad4db-1ab5-4228-9b23-695f11a33bed None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:55.185748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 672/1354] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:55.191209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734be0bb-61f7-4543-a4e2-d97d4a3e0cac None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:55.191643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734be0bb-61f7-4543-a4e2-d97d4a3e0cac None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:55.191757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734be0bb-61f7-4543-a4e2-d97d4a3e0cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:55.266577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-734be0bb-61f7-4543-a4e2-d97d4a3e0cac None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:55.268187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-734be0bb-61f7-4543-a4e2-d97d4a3e0cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:55.369246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 683/1355] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:55.671352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afc736-aa97-41fc-a3a0-6a12b05bc32e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:55.671752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afc736-aa97-41fc-a3a0-6a12b05bc32e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:55.672142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afc736-aa97-41fc-a3a0-6a12b05bc32e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:55.673736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 673/1356] 10.222.0.52 () {54 vars in 961 bytes} [Thu Oct 6 10:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:55.682901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8c23d-eda1-4ef9-bfa0-54080d919b56 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:55.683428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8c23d-eda1-4ef9-bfa0-54080d919b56 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:55.683904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8c23d-eda1-4ef9-bfa0-54080d919b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:55.723658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:55.789394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-42d8c23d-eda1-4ef9-bfa0-54080d919b56 None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:55.790507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-42d8c23d-eda1-4ef9-bfa0-54080d919b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:55.921763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 684/1357] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Oct 6 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:55.929703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afc736-aa97-41fc-a3a0-6a12b05bc32e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:55.931201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d7afc736-aa97-41fc-a3a0-6a12b05bc32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:56.091567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afc736-aa97-41fc-a3a0-6a12b05bc32e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:56.169718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:56.213094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e0e87-46ce-4f10-bfc2-8ef790c65de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:56.213692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e0e87-46ce-4f10-bfc2-8ef790c65de6 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:56.213974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e0e87-46ce-4f10-bfc2-8ef790c65de6 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:56.214248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e0e87-46ce-4f10-bfc2-8ef790c65de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:56.215871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-715e0e87-46ce-4f10-bfc2-8ef790c65de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:56.320622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e0e87-46ce-4f10-bfc2-8ef790c65de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:56.322294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e0e87-46ce-4f10-bfc2-8ef790c65de6 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:56.356095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 674/1358] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:56:58.770782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca0506-9ac6-40d5-a54b-ea7fb5532faf None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:58.771323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca0506-9ac6-40d5-a54b-ea7fb5532faf None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:58.771323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca0506-9ac6-40d5-a54b-ea7fb5532faf None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:58.777898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 685/1359] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:56:58.778881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f49c-3dcc-4da0-88ab-e677be3ca4e5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:58.778881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f49c-3dcc-4da0-88ab-e677be3ca4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:58.779662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f49c-3dcc-4da0-88ab-e677be3ca4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:58.852796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d975f49c-3dcc-4da0-88ab-e677be3ca4e5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:56:58.854092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d975f49c-3dcc-4da0-88ab-e677be3ca4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:58.892123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:58.954806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 675/1360] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:56:59.254173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca0506-9ac6-40d5-a54b-ea7fb5532faf None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:56:59.255727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-50ca0506-9ac6-40d5-a54b-ea7fb5532faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:59.334405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:59.446074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca0506-9ac6-40d5-a54b-ea7fb5532faf None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:56:59.567990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7f57-439c-4a7b-80d4-4097b13fadeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:56:59.568596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7f57-439c-4a7b-80d4-4097b13fadeb service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:56:59.568870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7f57-439c-4a7b-80d4-4097b13fadeb service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:56:59.569140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7f57-439c-4a7b-80d4-4097b13fadeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:56:59.570860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1c1c7f57-439c-4a7b-80d4-4097b13fadeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:56:59.595489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:56:59.669719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7f57-439c-4a7b-80d4-4097b13fadeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:56:59.671342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7f57-439c-4a7b-80d4-4097b13fadeb service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:56:59.690279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 686/1361] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:02.208058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4135-a13d-4e46-b502-21180f32a37f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:02.208500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4135-a13d-4e46-b502-21180f32a37f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:02.208611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4135-a13d-4e46-b502-21180f32a37f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:02.209975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 676/1362] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:02.216029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f06c54-8cf1-4188-82bd-1741db6775fe None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:02.216464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f06c54-8cf1-4188-82bd-1741db6775fe None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:02.216636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f06c54-8cf1-4188-82bd-1741db6775fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:02.289163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-21f06c54-8cf1-4188-82bd-1741db6775fe None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:02.290381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-21f06c54-8cf1-4188-82bd-1741db6775fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:02.395304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:02.407867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 687/1363] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:02.698493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c07d-8a45-4f4e-ab4a-6da780e808d2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:02.698781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c07d-8a45-4f4e-ab4a-6da780e808d2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:02.699086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c07d-8a45-4f4e-ab4a-6da780e808d2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:02.700617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 677/1364] 10.222.0.52 () {54 vars in 961 bytes} [Thu Oct 6 10:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:02.705993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec407-8aac-435b-bd18-252a8d9cb49e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:02.706514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec407-8aac-435b-bd18-252a8d9cb49e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:02.706797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec407-8aac-435b-bd18-252a8d9cb49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:02.792069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fa8ec407-8aac-435b-bd18-252a8d9cb49e None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:02.793507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fa8ec407-8aac-435b-bd18-252a8d9cb49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:02.918203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 688/1365] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Oct 6 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:02.926172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c07d-8a45-4f4e-ab4a-6da780e808d2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:02.927695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0a85c07d-8a45-4f4e-ab4a-6da780e808d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:03.084916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c07d-8a45-4f4e-ab4a-6da780e808d2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:03.108285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:03.215365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f73fc1-827e-4ae9-a81e-7689ebf53954 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:03.216107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f73fc1-827e-4ae9-a81e-7689ebf53954 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:03.216357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f73fc1-827e-4ae9-a81e-7689ebf53954 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:03.216567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f73fc1-827e-4ae9-a81e-7689ebf53954 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:03.218189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-14f73fc1-827e-4ae9-a81e-7689ebf53954 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:03.319703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f73fc1-827e-4ae9-a81e-7689ebf53954 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:03.321603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f73fc1-827e-4ae9-a81e-7689ebf53954 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:03.341398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 678/1366] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 10:57:03.361743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282cc2d-9180-4bef-aeb6-9757d972a5b3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:03.362301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282cc2d-9180-4bef-aeb6-9757d972a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:03.362301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282cc2d-9180-4bef-aeb6-9757d972a5b3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:03.363806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 689/1367] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:03.369175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a07bac-7f2f-4d0a-ab69-50f6fafb97e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:03.369479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a07bac-7f2f-4d0a-ab69-50f6fafb97e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:03.369770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a07bac-7f2f-4d0a-ab69-50f6fafb97e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:03.411661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:03.453103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d9a07bac-7f2f-4d0a-ab69-50f6fafb97e9 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:03.454397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d9a07bac-7f2f-4d0a-ab69-50f6fafb97e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:03.578640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 679/1368] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:03.931556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ede136-1377-4676-bf36-aac76fe99d9b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:03.931871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ede136-1377-4676-bf36-aac76fe99d9b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:03.932131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ede136-1377-4676-bf36-aac76fe99d9b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:03.933792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 690/1369] 10.222.0.52 () {54 vars in 963 bytes} [Thu Oct 6 10:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:03.940149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4505b-6297-4613-b8c0-d541a4af43eb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:03.940424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4505b-6297-4613-b8c0-d541a4af43eb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:03.940700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4505b-6297-4613-b8c0-d541a4af43eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:04.026972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-04b4505b-6297-4613-b8c0-d541a4af43eb None None] MFA Rules not processed for user `6ecccd27dfa04cf985c9fd632ce2fe60`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:04.028644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-04b4505b-6297-4613-b8c0-d541a4af43eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:04.087608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:04.120591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 680/1370] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Oct 6 10:57:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:04.128839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ede136-1377-4676-bf36-aac76fe99d9b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:04.130845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e1ede136-1377-4676-bf36-aac76fe99d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:04.189585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:04.249767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ede136-1377-4676-bf36-aac76fe99d9b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:04.338576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9457f7-8983-4074-bd5f-4d4ff50c374c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:04.339530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9457f7-8983-4074-bd5f-4d4ff50c374c service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:04.339978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9457f7-8983-4074-bd5f-4d4ff50c374c service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:04.340266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9457f7-8983-4074-bd5f-4d4ff50c374c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:04.342182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1f9457f7-8983-4074-bd5f-4d4ff50c374c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:04.467261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:04.483981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9457f7-8983-4074-bd5f-4d4ff50c374c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:04.485447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9457f7-8983-4074-bd5f-4d4ff50c374c service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:04.507682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 691/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:05.648689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c51b-dc3b-4ab7-9ad9-6bc97676e7f1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:05.649231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c51b-dc3b-4ab7-9ad9-6bc97676e7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:05.649339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c51b-dc3b-4ab7-9ad9-6bc97676e7f1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:05.650684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 681/1372] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:05.656234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de29f9-4e5d-4225-9bdf-0bb32210e6d2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:05.656663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de29f9-4e5d-4225-9bdf-0bb32210e6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:05.656777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de29f9-4e5d-4225-9bdf-0bb32210e6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:05.729830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-82de29f9-4e5d-4225-9bdf-0bb32210e6d2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:05.731044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-82de29f9-4e5d-4225-9bdf-0bb32210e6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:05.836530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 692/1373] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:06.129407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c51b-dc3b-4ab7-9ad9-6bc97676e7f1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:06.131030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4ee8c51b-dc3b-4ab7-9ad9-6bc97676e7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:06.283713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c51b-dc3b-4ab7-9ad9-6bc97676e7f1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:06.337102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:06.412953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04bb21-9584-4efc-8874-56dfd2b93a18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:06.414178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04bb21-9584-4efc-8874-56dfd2b93a18 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:06.414453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04bb21-9584-4efc-8874-56dfd2b93a18 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:06.414727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04bb21-9584-4efc-8874-56dfd2b93a18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:06.416365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f04bb21-9584-4efc-8874-56dfd2b93a18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:06.509527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04bb21-9584-4efc-8874-56dfd2b93a18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:06.511127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04bb21-9584-4efc-8874-56dfd2b93a18 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:06.530766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 682/1374] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:08.894828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f2cb7-adeb-4a4e-951b-8baa07f0f33f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:08.895364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f2cb7-adeb-4a4e-951b-8baa07f0f33f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:08.895364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f2cb7-adeb-4a4e-951b-8baa07f0f33f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:08.896897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 693/1375] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:08.902221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1c588-e495-49ca-952a-b232086a3397 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:08.902887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1c588-e495-49ca-952a-b232086a3397 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:08.903002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1c588-e495-49ca-952a-b232086a3397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:08.970019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cad1c588-e495-49ca-952a-b232086a3397 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:08.971209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cad1c588-e495-49ca-952a-b232086a3397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:08.979487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:09.080108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 683/1376] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:09.367861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f2cb7-adeb-4a4e-951b-8baa07f0f33f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:09.369386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e36f2cb7-adeb-4a4e-951b-8baa07f0f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:09.435613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:09.526550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f2cb7-adeb-4a4e-951b-8baa07f0f33f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:09.645928 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5c51b-b282-4ea9-8352-9a9f8b1a7856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:09.646445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5c51b-b282-4ea9-8352-9a9f8b1a7856 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:09.646717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5c51b-b282-4ea9-8352-9a9f8b1a7856 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:09.646982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5c51b-b282-4ea9-8352-9a9f8b1a7856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:09.650423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-10e5c51b-b282-4ea9-8352-9a9f8b1a7856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:09.669924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:09.747425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5c51b-b282-4ea9-8352-9a9f8b1a7856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:09.749051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5c51b-b282-4ea9-8352-9a9f8b1a7856 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:09.768104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 694/1377] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:12.020060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1a66-9f8c-413d-b2e2-09fa91bac102 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:12.020060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1a66-9f8c-413d-b2e2-09fa91bac102 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:12.020060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1a66-9f8c-413d-b2e2-09fa91bac102 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:12.021041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 684/1378] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:12.040342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925aac9-a375-4bca-8a2a-95e3770d4ef9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:12.040769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925aac9-a375-4bca-8a2a-95e3770d4ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:12.041005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925aac9-a375-4bca-8a2a-95e3770d4ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:12.129830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9925aac9-a375-4bca-8a2a-95e3770d4ef9 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:12.131035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9925aac9-a375-4bca-8a2a-95e3770d4ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:12.231695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:12.238182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 695/1379] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:12.527894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d1a66-9f8c-413d-b2e2-09fa91bac102 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:12.530094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db8d1a66-9f8c-413d-b2e2-09fa91bac102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:12.672784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:12.684468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d1a66-9f8c-413d-b2e2-09fa91bac102 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:12.833424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc9d06-c342-41fb-aad7-0eeb52be211f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:12.834075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9d06-c342-41fb-aad7-0eeb52be211f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:12.834357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9d06-c342-41fb-aad7-0eeb52be211f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:12.834631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9d06-c342-41fb-aad7-0eeb52be211f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:12.836347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2cfc9d06-c342-41fb-aad7-0eeb52be211f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:12.950314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc9d06-c342-41fb-aad7-0eeb52be211f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:12.952761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc9d06-c342-41fb-aad7-0eeb52be211f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:12.964588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:12.973626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 685/1380] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:15.554853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc7be-3ebf-4d4b-ab42-479d2e7ff6c4 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:15.555485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc7be-3ebf-4d4b-ab42-479d2e7ff6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:15.555485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc7be-3ebf-4d4b-ab42-479d2e7ff6c4 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:15.556772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 696/1381] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:15.563524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127318d7-c62f-43d0-9773-3785efc6a567 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:15.563998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127318d7-c62f-43d0-9773-3785efc6a567 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:15.564122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127318d7-c62f-43d0-9773-3785efc6a567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:15.634247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-127318d7-c62f-43d0-9773-3785efc6a567 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:15.635473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-127318d7-c62f-43d0-9773-3785efc6a567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:15.736904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 686/1382] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:16.123810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc7be-3ebf-4d4b-ab42-479d2e7ff6c4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:16.125475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cb7fc7be-3ebf-4d4b-ab42-479d2e7ff6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:16.278339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc7be-3ebf-4d4b-ab42-479d2e7ff6c4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:16.361108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:16.418235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf280c-ec9e-4c2d-ae9c-551181da4d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:16.418845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf280c-ec9e-4c2d-ae9c-551181da4d43 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:16.419120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf280c-ec9e-4c2d-ae9c-551181da4d43 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:16.419385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf280c-ec9e-4c2d-ae9c-551181da4d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:16.421022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-44bf280c-ec9e-4c2d-ae9c-551181da4d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:16.513395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf280c-ec9e-4c2d-ae9c-551181da4d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:16.514998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf280c-ec9e-4c2d-ae9c-551181da4d43 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:16.533419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 697/1383] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 10:57:18.785469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f04fe-06ae-47a6-8b23-78bae75c708a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:18.786017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f04fe-06ae-47a6-8b23-78bae75c708a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:18.786017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f04fe-06ae-47a6-8b23-78bae75c708a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:18.787363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 687/1384] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:18.793227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcd698-03ad-4ebd-b70b-f81d797b6cd7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:18.793683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcd698-03ad-4ebd-b70b-f81d797b6cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:18.793859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcd698-03ad-4ebd-b70b-f81d797b6cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:18.860079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c4fcd698-03ad-4ebd-b70b-f81d797b6cd7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:18.861287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c4fcd698-03ad-4ebd-b70b-f81d797b6cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:18.901609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:18.965181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 698/1385] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:19.259239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f04fe-06ae-47a6-8b23-78bae75c708a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:19.260763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bc8f04fe-06ae-47a6-8b23-78bae75c708a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:19.338368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:19.410024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f04fe-06ae-47a6-8b23-78bae75c708a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:19.525628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88c313-925d-4695-9e10-135aa9c7cbd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:19.526144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88c313-925d-4695-9e10-135aa9c7cbd2 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:19.526423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88c313-925d-4695-9e10-135aa9c7cbd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:19.526694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88c313-925d-4695-9e10-135aa9c7cbd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:19.528606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d88c313-925d-4695-9e10-135aa9c7cbd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:19.567867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:19.620945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88c313-925d-4695-9e10-135aa9c7cbd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:19.622574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88c313-925d-4695-9e10-135aa9c7cbd2 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:19.640861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 688/1386] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:19.837563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80301aeb-9213-4560-acc4-452803292e92 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:19.838138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80301aeb-9213-4560-acc4-452803292e92 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:19.838138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80301aeb-9213-4560-acc4-452803292e92 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:19.840000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 699/1387] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:19.846022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28ca1f-e38e-4d90-bbaf-f742125c6c0d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:19.846587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28ca1f-e38e-4d90-bbaf-f742125c6c0d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:19.846587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28ca1f-e38e-4d90-bbaf-f742125c6c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:19.918097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6f28ca1f-e38e-4d90-bbaf-f742125c6c0d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:19.919325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6f28ca1f-e38e-4d90-bbaf-f742125c6c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:20.014322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 689/1388] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:20.313844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17babaa4-34f5-4e28-9aa6-615c5defeac3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:20.314166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17babaa4-34f5-4e28-9aa6-615c5defeac3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:20.314469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17babaa4-34f5-4e28-9aa6-615c5defeac3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:20.315924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 700/1389] 10.222.0.52 () {54 vars in 962 bytes} [Thu Oct 6 10:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:20.321814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783a1cb-d018-4ece-b59b-8fdab34476ef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:20.322171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783a1cb-d018-4ece-b59b-8fdab34476ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:20.322515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783a1cb-d018-4ece-b59b-8fdab34476ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:20.339007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:20.405926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4783a1cb-d018-4ece-b59b-8fdab34476ef None None] MFA Rules not processed for user `9f85882ffc0447baaf3e00554f6b4ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:20.407134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4783a1cb-d018-4ece-b59b-8fdab34476ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:20.482404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 690/1390] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Oct 6 10:57:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:20.491682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17babaa4-34f5-4e28-9aa6-615c5defeac3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:20.493679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-17babaa4-34f5-4e28-9aa6-615c5defeac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:20.603722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17babaa4-34f5-4e28-9aa6-615c5defeac3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:20.663984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:20.688355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63115a-0b4f-496f-9ce3-ccc698cf4712 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:20.688962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63115a-0b4f-496f-9ce3-ccc698cf4712 service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:20.689237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63115a-0b4f-496f-9ce3-ccc698cf4712 service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:20.689504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63115a-0b4f-496f-9ce3-ccc698cf4712 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:20.691082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4e63115a-0b4f-496f-9ce3-ccc698cf4712 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:20.800302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63115a-0b4f-496f-9ce3-ccc698cf4712 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:20.801664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63115a-0b4f-496f-9ce3-ccc698cf4712 service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:20.820386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 701/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:21.559813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2684cf-d9e3-4b06-880a-89365596c7cc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:21.560320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2684cf-d9e3-4b06-880a-89365596c7cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:21.560520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2684cf-d9e3-4b06-880a-89365596c7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:21.627682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8d2684cf-d9e3-4b06-880a-89365596c7cc None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:21.628915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8d2684cf-d9e3-4b06-880a-89365596c7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:21.684052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:21.717161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 691/1392] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:21.764833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8013955-13cb-4308-8a5b-1c75b941be68 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:21.765755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8013955-13cb-4308-8a5b-1c75b941be68 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:21.766031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8013955-13cb-4308-8a5b-1c75b941be68 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:21.767533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 702/1393] 10.222.0.52 () {54 vars in 943 bytes} [Thu Oct 6 10:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:21.775372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca77a0f-8eb3-45e9-b0d5-28f95a7c08ef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:21.775372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca77a0f-8eb3-45e9-b0d5-28f95a7c08ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:21.775551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca77a0f-8eb3-45e9-b0d5-28f95a7c08ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:21.862576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2ca77a0f-8eb3-45e9-b0d5-28f95a7c08ef None None] MFA Rules not processed for user `df5aa06f9803465bb102482d351adeac`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:21.863973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2ca77a0f-8eb3-45e9-b0d5-28f95a7c08ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:21.884383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04d531-4bbd-4778-aeb1-4608854532a1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:21.884736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04d531-4bbd-4778-aeb1-4608854532a1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:21.884935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04d531-4bbd-4778-aeb1-4608854532a1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:21.886325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 703/1394] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 10:57:21.891501 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:21.891780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:21.892090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:21.973247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:21.974482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:21.990089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 692/1395] 10.222.0.52 () {58 vars in 1040 bytes} [Thu Oct 6 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 10:57:22.000391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca77a0f-8eb3-45e9-b0d5-28f95a7c08ef None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:22.010951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca77a0f-8eb3-45e9-b0d5-28f95a7c08ef None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:22.024680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:22.038269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:22.087313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 704/1396] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 10:57:22.135286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd087e86-e0a8-4c36-ba01-68bb5b223d73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:22.135926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd087e86-e0a8-4c36-ba01-68bb5b223d73 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:22.136207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd087e86-e0a8-4c36-ba01-68bb5b223d73 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:22.136477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd087e86-e0a8-4c36-ba01-68bb5b223d73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:22.138154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd087e86-e0a8-4c36-ba01-68bb5b223d73 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:22.221560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd087e86-e0a8-4c36-ba01-68bb5b223d73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:22.223154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd087e86-e0a8-4c36-ba01-68bb5b223d73 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:22.231627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 693/1397] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 10:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 06 10:57:22.383115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:22.384457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:22.511466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388dd47-bd64-43b2-a668-a69ed15c2ff1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:22.573566 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:22.642231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf99a27-8be9-4104-804a-00e12f0efc36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:22.642852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf99a27-8be9-4104-804a-00e12f0efc36 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:22.643132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf99a27-8be9-4104-804a-00e12f0efc36 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:22.643409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf99a27-8be9-4104-804a-00e12f0efc36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:22.645275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ecf99a27-8be9-4104-804a-00e12f0efc36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:22.748288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf99a27-8be9-4104-804a-00e12f0efc36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:22.750190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf99a27-8be9-4104-804a-00e12f0efc36 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:22.776130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 705/1398] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:25.012521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693d4b-bbd0-4899-8f34-7fbf565a3fa1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:25.013134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693d4b-bbd0-4899-8f34-7fbf565a3fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:25.013134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693d4b-bbd0-4899-8f34-7fbf565a3fa1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:25.016209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 694/1399] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:25.019753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e24339-b3ba-454b-b8a8-aaaf2cdc678c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:25.020247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e24339-b3ba-454b-b8a8-aaaf2cdc678c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:25.020332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e24339-b3ba-454b-b8a8-aaaf2cdc678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:25.095453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c4e24339-b3ba-454b-b8a8-aaaf2cdc678c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:25.096806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c4e24339-b3ba-454b-b8a8-aaaf2cdc678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:25.114163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:25.211770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 706/1400] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:25.508981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693d4b-bbd0-4899-8f34-7fbf565a3fa1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:25.510528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f693d4b-bbd0-4899-8f34-7fbf565a3fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:25.580539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:25.681758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693d4b-bbd0-4899-8f34-7fbf565a3fa1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:25.827105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22895639-163f-4135-ba41-9401d9ac50d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:25.827720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895639-163f-4135-ba41-9401d9ac50d4 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:25.828032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895639-163f-4135-ba41-9401d9ac50d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:25.828308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895639-163f-4135-ba41-9401d9ac50d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:25.830136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-22895639-163f-4135-ba41-9401d9ac50d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:25.851750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:25.972433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22895639-163f-4135-ba41-9401d9ac50d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:25.974708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22895639-163f-4135-ba41-9401d9ac50d4 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:25.997816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 695/1401] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:28.312542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2c9e-7b04-4ec5-a34b-b215e3ca2a7c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:28.313129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2c9e-7b04-4ec5-a34b-b215e3ca2a7c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:28.313129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2c9e-7b04-4ec5-a34b-b215e3ca2a7c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:28.314467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 707/1402] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:28.319734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef84c-02a0-4564-982f-1e1fbe3a61c4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:28.320778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef84c-02a0-4564-982f-1e1fbe3a61c4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:28.320912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef84c-02a0-4564-982f-1e1fbe3a61c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:28.399692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-68eef84c-02a0-4564-982f-1e1fbe3a61c4 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:28.401546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68eef84c-02a0-4564-982f-1e1fbe3a61c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:28.505496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:28.521551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 696/1403] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:28.825144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c2c9e-7b04-4ec5-a34b-b215e3ca2a7c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:28.826852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-939c2c9e-7b04-4ec5-a34b-b215e3ca2a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:28.984081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:28.987036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c2c9e-7b04-4ec5-a34b-b215e3ca2a7c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:29.111493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa777bef-7c3e-4433-ae10-e6c66d9a4b9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:29.112171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa777bef-7c3e-4433-ae10-e6c66d9a4b9e service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:29.112459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa777bef-7c3e-4433-ae10-e6c66d9a4b9e service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:29.112732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa777bef-7c3e-4433-ae10-e6c66d9a4b9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:29.114336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aa777bef-7c3e-4433-ae10-e6c66d9a4b9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:29.217532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa777bef-7c3e-4433-ae10-e6c66d9a4b9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:29.219154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa777bef-7c3e-4433-ae10-e6c66d9a4b9e service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:29.237683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 708/1404] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:31.505956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b6f3-4d95-40a2-81a8-541162978904 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:31.506524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b6f3-4d95-40a2-81a8-541162978904 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:31.506524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b6f3-4d95-40a2-81a8-541162978904 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:31.508531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 697/1405] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:31.514413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a9e8-bfe3-4e16-a51b-b9a5fdc1abe2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:31.514931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a9e8-bfe3-4e16-a51b-b9a5fdc1abe2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:31.515024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a9e8-bfe3-4e16-a51b-b9a5fdc1abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:31.519424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:31.588460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b636a9e8-bfe3-4e16-a51b-b9a5fdc1abe2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:31.589704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b636a9e8-bfe3-4e16-a51b-b9a5fdc1abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:31.699894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 709/1406] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:31.991074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b6f3-4d95-40a2-81a8-541162978904 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:31.993071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ec44b6f3-4d95-40a2-81a8-541162978904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:32.155776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b6f3-4d95-40a2-81a8-541162978904 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:32.225641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:32.282831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d2d72-ca4e-4594-a5aa-0fb45e799335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:32.283445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d2d72-ca4e-4594-a5aa-0fb45e799335 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:32.283725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d2d72-ca4e-4594-a5aa-0fb45e799335 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:32.284033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d2d72-ca4e-4594-a5aa-0fb45e799335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:32.285650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-157d2d72-ca4e-4594-a5aa-0fb45e799335 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:32.384498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d2d72-ca4e-4594-a5aa-0fb45e799335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:32.386125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d2d72-ca4e-4594-a5aa-0fb45e799335 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:32.407349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 698/1407] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:41.698388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=64 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:42.405041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=64 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:45.189157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f592199-fd4b-4410-b63c-f080ffa9d88c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:45.189730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f592199-fd4b-4410-b63c-f080ffa9d88c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:45.189730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f592199-fd4b-4410-b63c-f080ffa9d88c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:45.191052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 710/1408] 10.222.0.52 () {54 vars in 918 bytes} [Thu Oct 6 10:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:45.196641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16081-48e6-44c8-8ad2-6fdaebbc34e5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:45.197082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16081-48e6-44c8-8ad2-6fdaebbc34e5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:45.197191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16081-48e6-44c8-8ad2-6fdaebbc34e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:45.274747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-57f16081-48e6-44c8-8ad2-6fdaebbc34e5 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:45.275989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-57f16081-48e6-44c8-8ad2-6fdaebbc34e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:45.382454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 699/1409] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Oct 6 10:57:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:45.396619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c254-f035-4216-b942-72aedd44f123 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:45.396908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c254-f035-4216-b942-72aedd44f123 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:45.397189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c254-f035-4216-b942-72aedd44f123 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:45.398717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 711/1410] 10.222.0.52 () {54 vars in 918 bytes} [Thu Oct 6 10:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:45.404109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcce2e-98d1-41bb-8f10-b3f7f0fe0f2e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:45.404402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcce2e-98d1-41bb-8f10-b3f7f0fe0f2e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:45.404683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcce2e-98d1-41bb-8f10-b3f7f0fe0f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:45.477860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:45.481155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-51fcce2e-98d1-41bb-8f10-b3f7f0fe0f2e None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:45.482942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-51fcce2e-98d1-41bb-8f10-b3f7f0fe0f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:45.599088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 700/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Oct 6 10:57:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:48.224558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f7dd1-ffd4-4b93-b7de-aa0eb4e448b5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:48.225105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f7dd1-ffd4-4b93-b7de-aa0eb4e448b5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:48.225105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f7dd1-ffd4-4b93-b7de-aa0eb4e448b5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:48.226588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 712/1412] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:48.231862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff722b-33ff-48c1-8963-2ebbee6052ed None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:48.232241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff722b-33ff-48c1-8963-2ebbee6052ed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:48.232427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff722b-33ff-48c1-8963-2ebbee6052ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:48.298440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5eff722b-33ff-48c1-8963-2ebbee6052ed None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:48.299654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5eff722b-33ff-48c1-8963-2ebbee6052ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:48.354698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:48.397290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 701/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:48.692786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f7dd1-ffd4-4b93-b7de-aa0eb4e448b5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:48.695297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ca5f7dd1-ffd4-4b93-b7de-aa0eb4e448b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:48.838853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f7dd1-ffd4-4b93-b7de-aa0eb4e448b5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:48.911229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:48.951515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401ef99-f672-443f-81d3-8b9dbcbf3222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:48.952707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ef99-f672-443f-81d3-8b9dbcbf3222 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:48.952992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ef99-f672-443f-81d3-8b9dbcbf3222 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:48.953269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401ef99-f672-443f-81d3-8b9dbcbf3222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:48.954860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0401ef99-f672-443f-81d3-8b9dbcbf3222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:49.046749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401ef99-f672-443f-81d3-8b9dbcbf3222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:49.048373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401ef99-f672-443f-81d3-8b9dbcbf3222 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:49.066817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 713/1414] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:50.221181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595c8fe-a06f-4e06-8166-2945f7316dd4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:50.221751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595c8fe-a06f-4e06-8166-2945f7316dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:50.221751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595c8fe-a06f-4e06-8166-2945f7316dd4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:50.223065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 702/1415] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:50.229898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b380ff-2adf-4957-a17e-f887a59021ac None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:50.230356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b380ff-2adf-4957-a17e-f887a59021ac None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:50.230465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b380ff-2adf-4957-a17e-f887a59021ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:50.306126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-84b380ff-2adf-4957-a17e-f887a59021ac None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:50.307344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-84b380ff-2adf-4957-a17e-f887a59021ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:50.333638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:50.403130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 714/1416] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:50.554788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595c8fe-a06f-4e06-8166-2945f7316dd4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:50.556324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4595c8fe-a06f-4e06-8166-2945f7316dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:50.699978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595c8fe-a06f-4e06-8166-2945f7316dd4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:50.740059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:50.812135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf35c91-4be9-4c58-8484-5838ce55a9d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:50.812666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35c91-4be9-4c58-8484-5838ce55a9d5 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:50.812945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35c91-4be9-4c58-8484-5838ce55a9d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:50.813216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35c91-4be9-4c58-8484-5838ce55a9d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:50.814810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eaf35c91-4be9-4c58-8484-5838ce55a9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:50.904584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf35c91-4be9-4c58-8484-5838ce55a9d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:50.906185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf35c91-4be9-4c58-8484-5838ce55a9d5 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:50.924400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 703/1417] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:52.072140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb69f-9db5-432c-bd0c-2173903a1e30 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:52.072691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb69f-9db5-432c-bd0c-2173903a1e30 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:52.072691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb69f-9db5-432c-bd0c-2173903a1e30 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:52.074050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 715/1418] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:52.080341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011734e7-b261-41a6-a5a7-b3057cd4d61b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:52.080785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011734e7-b261-41a6-a5a7-b3057cd4d61b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:52.080891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011734e7-b261-41a6-a5a7-b3057cd4d61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:52.136020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:52.158260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-011734e7-b261-41a6-a5a7-b3057cd4d61b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:52.159462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-011734e7-b261-41a6-a5a7-b3057cd4d61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:52.256212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 704/1419] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:52.408773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fb69f-9db5-432c-bd0c-2173903a1e30 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:52.410262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ac0fb69f-9db5-432c-bd0c-2173903a1e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:52.556779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fb69f-9db5-432c-bd0c-2173903a1e30 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:52.568963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:52.670250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abab62-eb9a-456e-b8c7-6b47a1c50768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:52.670775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abab62-eb9a-456e-b8c7-6b47a1c50768 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:52.671053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abab62-eb9a-456e-b8c7-6b47a1c50768 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:52.671323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abab62-eb9a-456e-b8c7-6b47a1c50768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:52.672945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-23abab62-eb9a-456e-b8c7-6b47a1c50768 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:52.762262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abab62-eb9a-456e-b8c7-6b47a1c50768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:52.763991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abab62-eb9a-456e-b8c7-6b47a1c50768 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:52.781894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 716/1420] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:53.892492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc350-c51b-4129-b8ab-03372d90549e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:53.893078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc350-c51b-4129-b8ab-03372d90549e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:53.893078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc350-c51b-4129-b8ab-03372d90549e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:53.894390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 705/1421] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:53.901205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6c2c2-51f5-492b-91a8-91758e14c080 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:53.901665 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6c2c2-51f5-492b-91a8-91758e14c080 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:53.902033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6c2c2-51f5-492b-91a8-91758e14c080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:53.922484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:53.979169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-25e6c2c2-51f5-492b-91a8-91758e14c080 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:53.980400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-25e6c2c2-51f5-492b-91a8-91758e14c080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:54.081221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 717/1422] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:54.235192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedc350-c51b-4129-b8ab-03372d90549e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:54.236726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6dedc350-c51b-4129-b8ab-03372d90549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:54.345376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:54.381099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedc350-c51b-4129-b8ab-03372d90549e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:54.491875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324ccac-3d06-441a-8eac-c4e863449c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:54.492423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324ccac-3d06-441a-8eac-c4e863449c60 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:54.492682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324ccac-3d06-441a-8eac-c4e863449c60 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:54.492946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324ccac-3d06-441a-8eac-c4e863449c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:54.494529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9324ccac-3d06-441a-8eac-c4e863449c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:54.570222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:54.583613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324ccac-3d06-441a-8eac-c4e863449c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:54.585273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324ccac-3d06-441a-8eac-c4e863449c60 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:54.602683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 706/1423] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:55.711184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a69b22-edd8-4516-9f39-4830022dfe44 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:55.711722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a69b22-edd8-4516-9f39-4830022dfe44 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:55.711722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a69b22-edd8-4516-9f39-4830022dfe44 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:55.713051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 718/1424] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:55.719453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb19d4d-e779-476e-8940-9e0bc79a7ce5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:55.719896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb19d4d-e779-476e-8940-9e0bc79a7ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:55.720055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb19d4d-e779-476e-8940-9e0bc79a7ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:55.794583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7fb19d4d-e779-476e-8940-9e0bc79a7ce5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:55.795769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7fb19d4d-e779-476e-8940-9e0bc79a7ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:55.889588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 707/1425] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:56.048114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a69b22-edd8-4516-9f39-4830022dfe44 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:56.049688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f4a69b22-edd8-4516-9f39-4830022dfe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:56.133614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:56.195820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a69b22-edd8-4516-9f39-4830022dfe44 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:56.312401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc1368-9368-49d4-87fa-852cc4a634e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:56.313290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1368-9368-49d4-87fa-852cc4a634e8 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:56.313762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1368-9368-49d4-87fa-852cc4a634e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:56.314218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1368-9368-49d4-87fa-852cc4a634e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:56.317085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c7dc1368-9368-49d4-87fa-852cc4a634e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:56.365825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:56.407917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc1368-9368-49d4-87fa-852cc4a634e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:56.409591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc1368-9368-49d4-87fa-852cc4a634e8 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:56.427342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 719/1426] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:57.472887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1beef-d472-46c3-be9e-30abf337ef8b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:57.473470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1beef-d472-46c3-be9e-30abf337ef8b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:57.473470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1beef-d472-46c3-be9e-30abf337ef8b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:57.474785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 708/1427] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:57.481133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c46db3-2025-41b2-94fb-7a87f94c830a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:57.481621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c46db3-2025-41b2-94fb-7a87f94c830a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:57.481726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c46db3-2025-41b2-94fb-7a87f94c830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:57.557545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b2c46db3-2025-41b2-94fb-7a87f94c830a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:57.558750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b2c46db3-2025-41b2-94fb-7a87f94c830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:57.654053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 720/1428] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:57.804697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1beef-d472-46c3-be9e-30abf337ef8b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:57.806209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-42c1beef-d472-46c3-be9e-30abf337ef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:57.855652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:57.952775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1beef-d472-46c3-be9e-30abf337ef8b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:58.067500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88096d31-b5c2-4078-8d66-0f145b7298c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:58.068130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88096d31-b5c2-4078-8d66-0f145b7298c3 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:58.068406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88096d31-b5c2-4078-8d66-0f145b7298c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:58.068675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88096d31-b5c2-4078-8d66-0f145b7298c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:58.070835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-88096d31-b5c2-4078-8d66-0f145b7298c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:58.077664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:58.159381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88096d31-b5c2-4078-8d66-0f145b7298c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:58.161024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88096d31-b5c2-4078-8d66-0f145b7298c3 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:58.180985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 709/1429] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:57:59.243237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba024a-13bb-4b5c-ab64-1732a9c70fc7 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:59.243813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba024a-13bb-4b5c-ab64-1732a9c70fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:59.243813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba024a-13bb-4b5c-ab64-1732a9c70fc7 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:59.245194 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 721/1430] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:57:59.251465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8338e-11aa-434e-a210-44cf3a59bb45 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:59.251919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8338e-11aa-434e-a210-44cf3a59bb45 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:59.252057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8338e-11aa-434e-a210-44cf3a59bb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:59.327852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-03b8338e-11aa-434e-a210-44cf3a59bb45 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:57:59.329083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-03b8338e-11aa-434e-a210-44cf3a59bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:59.398010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:59.427201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 710/1431] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:57:59.578894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba024a-13bb-4b5c-ab64-1732a9c70fc7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:57:59.580390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-29ba024a-13bb-4b5c-ab64-1732a9c70fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:59.598279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:59.726281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba024a-13bb-4b5c-ab64-1732a9c70fc7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:57:59.814401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:57:59.840498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc627cb8-2039-4678-a0f7-12fcdc858718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:57:59.841059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc627cb8-2039-4678-a0f7-12fcdc858718 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:57:59.841338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc627cb8-2039-4678-a0f7-12fcdc858718 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:57:59.841609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc627cb8-2039-4678-a0f7-12fcdc858718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:57:59.843203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cc627cb8-2039-4678-a0f7-12fcdc858718 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:57:59.932102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc627cb8-2039-4678-a0f7-12fcdc858718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:57:59.933703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc627cb8-2039-4678-a0f7-12fcdc858718 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:57:59.951749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 722/1432] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:01.034037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7abc8-d71e-4fd4-93e4-2544417d55e9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:01.034728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7abc8-d71e-4fd4-93e4-2544417d55e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:01.034728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7abc8-d71e-4fd4-93e4-2544417d55e9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:01.036070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 711/1433] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:01.043053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e39c5-456a-41d1-a0e3-8e9304189f67 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:01.043490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e39c5-456a-41d1-a0e3-8e9304189f67 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:01.043595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e39c5-456a-41d1-a0e3-8e9304189f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:01.122423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-233e39c5-456a-41d1-a0e3-8e9304189f67 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:01.123656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-233e39c5-456a-41d1-a0e3-8e9304189f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:01.165683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:01.220454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 723/1434] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:01.374459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7abc8-d71e-4fd4-93e4-2544417d55e9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:01.375926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5da7abc8-d71e-4fd4-93e4-2544417d55e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:01.520935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7abc8-d71e-4fd4-93e4-2544417d55e9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:01.576353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:01.635228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591389a-149f-4e93-9951-390a8417f87c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:01.635756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591389a-149f-4e93-9951-390a8417f87c service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:01.636062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591389a-149f-4e93-9951-390a8417f87c service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:01.636336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591389a-149f-4e93-9951-390a8417f87c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:01.637939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8591389a-149f-4e93-9951-390a8417f87c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:01.728494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591389a-149f-4e93-9951-390a8417f87c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:01.730375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591389a-149f-4e93-9951-390a8417f87c service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:01.747980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 712/1435] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:02.797753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0516ed-a69e-4ace-9954-0d810261cc66 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:02.798297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0516ed-a69e-4ace-9954-0d810261cc66 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:02.798415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0516ed-a69e-4ace-9954-0d810261cc66 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:02.799670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 724/1436] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 10:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:02.805938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379958be-cd92-4f67-a2b2-7fbe08a878cd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:02.806412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379958be-cd92-4f67-a2b2-7fbe08a878cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:02.806569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379958be-cd92-4f67-a2b2-7fbe08a878cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:02.880630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-379958be-cd92-4f67-a2b2-7fbe08a878cd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:02.882495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-379958be-cd92-4f67-a2b2-7fbe08a878cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:02.884507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:02.979104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 713/1437] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:03.130540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0516ed-a69e-4ace-9954-0d810261cc66 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:03.132102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4a0516ed-a69e-4ace-9954-0d810261cc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:03.278431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0516ed-a69e-4ace-9954-0d810261cc66 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:03.305220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:03.390254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c6cf7-6510-4aad-b87f-39eced3c4ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:03.391149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c6cf7-6510-4aad-b87f-39eced3c4ff2 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:03.391149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c6cf7-6510-4aad-b87f-39eced3c4ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:03.391350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c6cf7-6510-4aad-b87f-39eced3c4ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:03.392905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-721c6cf7-6510-4aad-b87f-39eced3c4ff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:03.484098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c6cf7-6510-4aad-b87f-39eced3c4ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:03.485719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c6cf7-6510-4aad-b87f-39eced3c4ff2 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:03.504013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 725/1438] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:05.610933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daa7d0-3c71-4917-b879-cea20e2dc57c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:05.611523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daa7d0-3c71-4917-b879-cea20e2dc57c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:05.611523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daa7d0-3c71-4917-b879-cea20e2dc57c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:05.612939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 714/1439] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:05.618572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1f04e-5d36-49f7-afad-0c694df53fa1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:05.619020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1f04e-5d36-49f7-afad-0c694df53fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:05.619127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1f04e-5d36-49f7-afad-0c694df53fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:05.660476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:05.686348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ccd1f04e-5d36-49f7-afad-0c694df53fa1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:05.687553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ccd1f04e-5d36-49f7-afad-0c694df53fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:05.792167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 726/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:06.084948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36daa7d0-3c71-4917-b879-cea20e2dc57c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:06.086519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-36daa7d0-3c71-4917-b879-cea20e2dc57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:06.189163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36daa7d0-3c71-4917-b879-cea20e2dc57c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:06.221543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:06.265132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f7151-6d81-4945-956c-009acda763ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:06.265736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f7151-6d81-4945-956c-009acda763ed service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:06.266013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f7151-6d81-4945-956c-009acda763ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:06.266284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f7151-6d81-4945-956c-009acda763ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:06.267905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c60f7151-6d81-4945-956c-009acda763ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:06.375499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f7151-6d81-4945-956c-009acda763ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:06.376887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f7151-6d81-4945-956c-009acda763ed service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:06.395761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 715/1441] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:08.697879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6edb6-38f4-44f5-a8f7-a445dd586e70 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:08.698417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6edb6-38f4-44f5-a8f7-a445dd586e70 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:08.698417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6edb6-38f4-44f5-a8f7-a445dd586e70 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:08.699700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 727/1442] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:08.704835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208f16-740b-4eca-8048-626f780dd334 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:08.705246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208f16-740b-4eca-8048-626f780dd334 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:08.705392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b208f16-740b-4eca-8048-626f780dd334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:08.771052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7b208f16-740b-4eca-8048-626f780dd334 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:08.772954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b208f16-740b-4eca-8048-626f780dd334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:08.785788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:08.869960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 716/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:09.189884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85ca1-ae89-4fb2-ac96-887ef40a18d0 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:09.190166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85ca1-ae89-4fb2-ac96-887ef40a18d0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:09.190439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85ca1-ae89-4fb2-ac96-887ef40a18d0 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:09.191783 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 728/1444] 10.222.0.52 () {54 vars in 964 bytes} [Thu Oct 6 10:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:09.199000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc258c-fa11-45d8-a7f8-33511a6a3aad None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:09.199280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc258c-fa11-45d8-a7f8-33511a6a3aad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:09.199549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc258c-fa11-45d8-a7f8-33511a6a3aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:09.290172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-00cc258c-fa11-45d8-a7f8-33511a6a3aad None None] MFA Rules not processed for user `b5c892640b3e409b9a99b4ca1d34a964`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:09.291386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-00cc258c-fa11-45d8-a7f8-33511a6a3aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:09.370924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 717/1445] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Oct 6 10:58:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:09.381145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85ca1-ae89-4fb2-ac96-887ef40a18d0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:09.382901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-13e85ca1-ae89-4fb2-ac96-887ef40a18d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:09.498108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85ca1-ae89-4fb2-ac96-887ef40a18d0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:09.512114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:09.578824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694eef1-0f5a-4723-80ac-02128a28248b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:09.579435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694eef1-0f5a-4723-80ac-02128a28248b service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:09.579713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694eef1-0f5a-4723-80ac-02128a28248b service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:09.580019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694eef1-0f5a-4723-80ac-02128a28248b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:09.581620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6694eef1-0f5a-4723-80ac-02128a28248b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:09.819932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694eef1-0f5a-4723-80ac-02128a28248b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:09.821293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694eef1-0f5a-4723-80ac-02128a28248b service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:09.839586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 729/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 10:58:12.755807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c9692-7266-48a5-8821-583d95b9e1b0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:12.756390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c9692-7266-48a5-8821-583d95b9e1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:12.756390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c9692-7266-48a5-8821-583d95b9e1b0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:12.757754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:12.763402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab865c7-b4bd-4458-a1b9-c10e59af11ba None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:12.763823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab865c7-b4bd-4458-a1b9-c10e59af11ba None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:12.763968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab865c7-b4bd-4458-a1b9-c10e59af11ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:12.817897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:12.836657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5ab865c7-b4bd-4458-a1b9-c10e59af11ba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:12.837848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5ab865c7-b4bd-4458-a1b9-c10e59af11ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:12.932679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:13.215423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c9692-7266-48a5-8821-583d95b9e1b0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:13.217021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5a5c9692-7266-48a5-8821-583d95b9e1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:13.225254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:13.321866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c9692-7266-48a5-8821-583d95b9e1b0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:13.393639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a52ea-e0f2-4831-ab91-042a9eb0c59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:13.394244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a52ea-e0f2-4831-ab91-042a9eb0c59a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:13.394520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a52ea-e0f2-4831-ab91-042a9eb0c59a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:13.394788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a52ea-e0f2-4831-ab91-042a9eb0c59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:13.396983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f4a52ea-e0f2-4831-ab91-042a9eb0c59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:13.456110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:13.505165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a52ea-e0f2-4831-ab91-042a9eb0c59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:13.507212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a52ea-e0f2-4831-ab91-042a9eb0c59a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:13.524583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 719/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:15.887186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49251cbc-8ba7-4142-953a-8bd6a69b744d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:15.887752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49251cbc-8ba7-4142-953a-8bd6a69b744d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:15.887752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49251cbc-8ba7-4142-953a-8bd6a69b744d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:15.889134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:15.894573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92923417-1067-4444-96b2-4fb2aefa2400 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:15.894999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92923417-1067-4444-96b2-4fb2aefa2400 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:15.895127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92923417-1067-4444-96b2-4fb2aefa2400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:15.961375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-92923417-1067-4444-96b2-4fb2aefa2400 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:15.962562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-92923417-1067-4444-96b2-4fb2aefa2400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:16.057587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:16.339448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49251cbc-8ba7-4142-953a-8bd6a69b744d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:16.341037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-49251cbc-8ba7-4142-953a-8bd6a69b744d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:16.446629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49251cbc-8ba7-4142-953a-8bd6a69b744d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:16.467757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:16.520648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee992da-460a-4ea6-be3c-0f3553288db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:16.521677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee992da-460a-4ea6-be3c-0f3553288db6 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:16.521960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee992da-460a-4ea6-be3c-0f3553288db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:16.522232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee992da-460a-4ea6-be3c-0f3553288db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:16.523814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6ee992da-460a-4ea6-be3c-0f3553288db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:16.632510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee992da-460a-4ea6-be3c-0f3553288db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:16.633958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee992da-460a-4ea6-be3c-0f3553288db6 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:16.652486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 732/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:18.926726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5355a9e-1e0c-43f8-9f62-5c16da940ae9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:18.927257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5355a9e-1e0c-43f8-9f62-5c16da940ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:18.927257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5355a9e-1e0c-43f8-9f62-5c16da940ae9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:18.928651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:18.933997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7efde7-94ab-44c9-882b-ecaf3e4329e7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:18.934454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7efde7-94ab-44c9-882b-ecaf3e4329e7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:18.934554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7efde7-94ab-44c9-882b-ecaf3e4329e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:18.997199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:19.000350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ec7efde7-94ab-44c9-882b-ecaf3e4329e7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:19.002006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ec7efde7-94ab-44c9-882b-ecaf3e4329e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:19.097227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:19.382010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5355a9e-1e0c-43f8-9f62-5c16da940ae9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:19.383548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b5355a9e-1e0c-43f8-9f62-5c16da940ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:19.402395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:19.488429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5355a9e-1e0c-43f8-9f62-5c16da940ae9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:19.679061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4244e-a99f-40dd-9b97-63f701038a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:19.679563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4244e-a99f-40dd-9b97-63f701038a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:19.679856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4244e-a99f-40dd-9b97-63f701038a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:19.680131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4244e-a99f-40dd-9b97-63f701038a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:19.681710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8ed4244e-a99f-40dd-9b97-63f701038a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:19.751030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:19.791301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4244e-a99f-40dd-9b97-63f701038a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:19.792753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4244e-a99f-40dd-9b97-63f701038a39 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:19.810325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 722/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 10:58:22.009128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986330-0876-4e8c-a6d2-87eed1a936c1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:22.009710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986330-0876-4e8c-a6d2-87eed1a936c1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:22.009710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986330-0876-4e8c-a6d2-87eed1a936c1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:22.011033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:22.016276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f32a8-da08-4515-b583-8b5807c313fb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:22.016691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f32a8-da08-4515-b583-8b5807c313fb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:22.016812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f32a8-da08-4515-b583-8b5807c313fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:22.082592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-115f32a8-da08-4515-b583-8b5807c313fb None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:22.084398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-115f32a8-da08-4515-b583-8b5807c313fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:22.186571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 10:58:22.469713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8986330-0876-4e8c-a6d2-87eed1a936c1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:22.471230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e8986330-0876-4e8c-a6d2-87eed1a936c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:22.574137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8986330-0876-4e8c-a6d2-87eed1a936c1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:22.603232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:22.648750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ff35e-43a9-4d1e-a6b2-610676b69392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:22.649278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ff35e-43a9-4d1e-a6b2-610676b69392 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:22.649555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ff35e-43a9-4d1e-a6b2-610676b69392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:22.649824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ff35e-43a9-4d1e-a6b2-610676b69392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:22.651649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-673ff35e-43a9-4d1e-a6b2-610676b69392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:22.758871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ff35e-43a9-4d1e-a6b2-610676b69392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:22.760350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ff35e-43a9-4d1e-a6b2-610676b69392 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:22.778332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 735/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:24.994916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7122457-a119-478c-a766-e14ab8a80028 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:24.995461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7122457-a119-478c-a766-e14ab8a80028 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:24.995461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7122457-a119-478c-a766-e14ab8a80028 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:24.996859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:25.002070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46b71-f18d-4f06-ac7b-60ea3c46de3a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:25.002481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46b71-f18d-4f06-ac7b-60ea3c46de3a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:25.002625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46b71-f18d-4f06-ac7b-60ea3c46de3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:25.069038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5fd46b71-f18d-4f06-ac7b-60ea3c46de3a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:25.070244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5fd46b71-f18d-4f06-ac7b-60ea3c46de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:25.078531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:25.166709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:25.455000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7122457-a119-478c-a766-e14ab8a80028 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:25.456591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b7122457-a119-478c-a766-e14ab8a80028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:25.485145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:25.563084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7122457-a119-478c-a766-e14ab8a80028 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:25.637818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7483213-da2d-49ef-9612-33abba58cefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:25.638421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483213-da2d-49ef-9612-33abba58cefe service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:25.638703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483213-da2d-49ef-9612-33abba58cefe service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:25.638975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483213-da2d-49ef-9612-33abba58cefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:25.640663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f7483213-da2d-49ef-9612-33abba58cefe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:25.717253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:25.750228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7483213-da2d-49ef-9612-33abba58cefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:25.751663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7483213-da2d-49ef-9612-33abba58cefe service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:25.769797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 725/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:27.979503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f907211-6f32-4a02-8cff-279d5fc261c3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:27.980216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f907211-6f32-4a02-8cff-279d5fc261c3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:27.980216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f907211-6f32-4a02-8cff-279d5fc261c3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:27.981471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:27.986907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd695-cd2d-41a0-ae6e-9a958a27bd2b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:27.987359 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd695-cd2d-41a0-ae6e-9a958a27bd2b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:27.987468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd695-cd2d-41a0-ae6e-9a958a27bd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:28.053605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e4bbd695-cd2d-41a0-ae6e-9a958a27bd2b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:28.054843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e4bbd695-cd2d-41a0-ae6e-9a958a27bd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:28.150160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:28.434619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f907211-6f32-4a02-8cff-279d5fc261c3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:28.436212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0f907211-6f32-4a02-8cff-279d5fc261c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:28.541304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f907211-6f32-4a02-8cff-279d5fc261c3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:28.579637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:28.616776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c7b1-7ed7-431b-8af7-a225098571c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:28.617378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c7b1-7ed7-431b-8af7-a225098571c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:28.617653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c7b1-7ed7-431b-8af7-a225098571c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:28.617920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c7b1-7ed7-431b-8af7-a225098571c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:28.619491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c5d5c7b1-7ed7-431b-8af7-a225098571c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:28.729569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c7b1-7ed7-431b-8af7-a225098571c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:28.730707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c7b1-7ed7-431b-8af7-a225098571c4 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:28.749750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 738/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:30.914714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ecacf-1cab-4562-aea7-ebb739e5c7c7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:30.915282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ecacf-1cab-4562-aea7-ebb739e5c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:30.915282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ecacf-1cab-4562-aea7-ebb739e5c7c7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:30.916653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:30.921860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a57f11-92e2-4bac-ac25-ff84ab66686d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:30.922259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a57f11-92e2-4bac-ac25-ff84ab66686d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:30.922404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a57f11-92e2-4bac-ac25-ff84ab66686d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:30.986995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-29a57f11-92e2-4bac-ac25-ff84ab66686d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:30.988216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-29a57f11-92e2-4bac-ac25-ff84ab66686d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:31.006799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:31.085134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:31.371798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ecacf-1cab-4562-aea7-ebb739e5c7c7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:31.373355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c22ecacf-1cab-4562-aea7-ebb739e5c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:31.410728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:31.476955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ecacf-1cab-4562-aea7-ebb739e5c7c7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:31.551172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcae4dc-e0d3-4edd-9cf3-3122ff9a3cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:31.551814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcae4dc-e0d3-4edd-9cf3-3122ff9a3cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:31.552128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcae4dc-e0d3-4edd-9cf3-3122ff9a3cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:31.552483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcae4dc-e0d3-4edd-9cf3-3122ff9a3cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:31.554085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddcae4dc-e0d3-4edd-9cf3-3122ff9a3cb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:31.636225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:31.663490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcae4dc-e0d3-4edd-9cf3-3122ff9a3cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:31.664950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcae4dc-e0d3-4edd-9cf3-3122ff9a3cb4 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:31.682342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 728/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:33.939729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe705bb2-48c7-440c-87dd-d428dacc0077 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:33.940320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe705bb2-48c7-440c-87dd-d428dacc0077 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:33.940320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe705bb2-48c7-440c-87dd-d428dacc0077 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:33.941691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 10:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:33.947217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74fd4f-6e45-4baf-a675-c8f83c365a7f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:33.947665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74fd4f-6e45-4baf-a675-c8f83c365a7f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:33.947768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74fd4f-6e45-4baf-a675-c8f83c365a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:34.012881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1a74fd4f-6e45-4baf-a675-c8f83c365a7f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:34.014103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1a74fd4f-6e45-4baf-a675-c8f83c365a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:34.109012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 10:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:34.395305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe705bb2-48c7-440c-87dd-d428dacc0077 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:34.397641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fe705bb2-48c7-440c-87dd-d428dacc0077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:34.498385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe705bb2-48c7-440c-87dd-d428dacc0077 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:34.545567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:34.571675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27490edd-7196-4460-a571-0047e567a8ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:34.572299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27490edd-7196-4460-a571-0047e567a8ed service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:34.572576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27490edd-7196-4460-a571-0047e567a8ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:34.572845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27490edd-7196-4460-a571-0047e567a8ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:34.574410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-27490edd-7196-4460-a571-0047e567a8ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:34.682462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27490edd-7196-4460-a571-0047e567a8ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:34.683912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27490edd-7196-4460-a571-0047e567a8ed service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:34.701510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 741/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:44.107490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:44.699338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=66 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:55.364979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0419e6-fea5-4059-9f5d-7d2fa79d0bd1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:55.365574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0419e6-fea5-4059-9f5d-7d2fa79d0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:55.365574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0419e6-fea5-4059-9f5d-7d2fa79d0bd1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:55.366876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 730/1471] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:55.372182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b9646-ea4f-417e-a7b5-b9ee59d66860 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:55.372645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b9646-ea4f-417e-a7b5-b9ee59d66860 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:55.372752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b9646-ea4f-417e-a7b5-b9ee59d66860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:55.437715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d88b9646-ea4f-417e-a7b5-b9ee59d66860 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:55.438897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d88b9646-ea4f-417e-a7b5-b9ee59d66860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:55.537545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 742/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:55.839583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421a54-b4e0-4d8b-a503-a74c5e0cdcb4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:55.839911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421a54-b4e0-4d8b-a503-a74c5e0cdcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:55.840186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421a54-b4e0-4d8b-a503-a74c5e0cdcb4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:55.841514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 731/1473] 10.222.0.52 () {54 vars in 962 bytes} [Thu Oct 6 10:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:55.846334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21046f-f06f-49eb-9091-e95101b74700 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:55.846644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21046f-f06f-49eb-9091-e95101b74700 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:55.846916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21046f-f06f-49eb-9091-e95101b74700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:55.923435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ad21046f-f06f-49eb-9091-e95101b74700 None None] MFA Rules not processed for user `9f85882ffc0447baaf3e00554f6b4ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:55.924687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ad21046f-f06f-49eb-9091-e95101b74700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:55.946095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:56.002081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 743/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Oct 6 10:58:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:56.009908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421a54-b4e0-4d8b-a503-a74c5e0cdcb4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:56.011407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e421a54-b4e0-4d8b-a503-a74c5e0cdcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:56.114312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421a54-b4e0-4d8b-a503-a74c5e0cdcb4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:56.187136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a40bdb-b8ae-48d3-9e48-9489e361f5ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:56.187730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a40bdb-b8ae-48d3-9e48-9489e361f5ff service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:56.188030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a40bdb-b8ae-48d3-9e48-9489e361f5ff service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:56.188300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a40bdb-b8ae-48d3-9e48-9489e361f5ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:56.189864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-07a40bdb-b8ae-48d3-9e48-9489e361f5ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:56.242594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:56.296594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a40bdb-b8ae-48d3-9e48-9489e361f5ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:56.298022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a40bdb-b8ae-48d3-9e48-9489e361f5ff service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:56.316067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 732/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:58:58.554158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111c493-7f63-41f1-9267-dec6f8f1a877 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:58.554714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111c493-7f63-41f1-9267-dec6f8f1a877 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:58.554714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111c493-7f63-41f1-9267-dec6f8f1a877 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:58.556220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 744/1476] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:58:58.561623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec353c09-44c9-47f3-a155-be60d715fdda None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:58.562111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec353c09-44c9-47f3-a155-be60d715fdda None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:58.562220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec353c09-44c9-47f3-a155-be60d715fdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:58.627170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ec353c09-44c9-47f3-a155-be60d715fdda None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:58:58.628624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ec353c09-44c9-47f3-a155-be60d715fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:58.727011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 733/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:58:59.010954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111c493-7f63-41f1-9267-dec6f8f1a877 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:58:59.012558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a111c493-7f63-41f1-9267-dec6f8f1a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:58:59.139523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111c493-7f63-41f1-9267-dec6f8f1a877 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:58:59.189079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:58:59.233149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b49f4-45ec-4906-ad12-ed5497991e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:58:59.233752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b49f4-45ec-4906-ad12-ed5497991e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:58:59.234032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b49f4-45ec-4906-ad12-ed5497991e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:58:59.234307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b49f4-45ec-4906-ad12-ed5497991e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:58:59.234943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b49f4-45ec-4906-ad12-ed5497991e3a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:58:59.236703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-198b49f4-45ec-4906-ad12-ed5497991e3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 10:58:59.236703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 10:58:59.236703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 10:58:59.237849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b49f4-45ec-4906-ad12-ed5497991e3a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:58:59.257437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 745/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Oct 6 10:58:59 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 10:59:01.429426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b18487-80dd-4fdb-99f4-784ffa77396e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:01.429963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b18487-80dd-4fdb-99f4-784ffa77396e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:01.429963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b18487-80dd-4fdb-99f4-784ffa77396e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:01.431324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 734/1479] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:59:01.436744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fb438-4a37-4201-be33-9e71b0d8b1f5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:01.437146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fb438-4a37-4201-be33-9e71b0d8b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:01.437291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fb438-4a37-4201-be33-9e71b0d8b1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:01.504584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a33fb438-4a37-4201-be33-9e71b0d8b1f5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:59:01.505779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a33fb438-4a37-4201-be33-9e71b0d8b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:01.600304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 746/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:59:01.896567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b18487-80dd-4fdb-99f4-784ffa77396e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:59:01.898214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8b18487-80dd-4fdb-99f4-784ffa77396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:01.908272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:02.042905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b18487-80dd-4fdb-99f4-784ffa77396e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:59:02.122413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:02.156574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99439b-e03a-4f76-aab2-78723f0fe873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:59:02.157106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99439b-e03a-4f76-aab2-78723f0fe873 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:02.157376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99439b-e03a-4f76-aab2-78723f0fe873 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:02.157640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99439b-e03a-4f76-aab2-78723f0fe873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:02.159193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb99439b-e03a-4f76-aab2-78723f0fe873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:02.247691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99439b-e03a-4f76-aab2-78723f0fe873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:59:02.249294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99439b-e03a-4f76-aab2-78723f0fe873 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:59:02.266981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 735/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 10:59:04.600088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024a01a-35e6-40d5-ac03-3b5fc39fe17e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:04.600623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024a01a-35e6-40d5-ac03-3b5fc39fe17e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:04.600623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024a01a-35e6-40d5-ac03-3b5fc39fe17e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:04.601919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 747/1482] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:59:04.607230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdd43f-b13b-4657-9cde-0ad99ce80a7e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:04.607662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdd43f-b13b-4657-9cde-0ad99ce80a7e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:04.607908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdd43f-b13b-4657-9cde-0ad99ce80a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:04.673450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bebdd43f-b13b-4657-9cde-0ad99ce80a7e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:59:04.674658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bebdd43f-b13b-4657-9cde-0ad99ce80a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:04.720418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:04.770711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 736/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:59:05.062153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024a01a-35e6-40d5-ac03-3b5fc39fe17e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:59:05.063698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7024a01a-35e6-40d5-ac03-3b5fc39fe17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:05.088150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:05.169259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024a01a-35e6-40d5-ac03-3b5fc39fe17e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:59:05.241989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae191b-3060-4f6b-b21c-da424aec5a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:59:05.242586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae191b-3060-4f6b-b21c-da424aec5a96 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:05.242859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae191b-3060-4f6b-b21c-da424aec5a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:05.243121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae191b-3060-4f6b-b21c-da424aec5a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:05.244715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-78ae191b-3060-4f6b-b21c-da424aec5a96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:05.317667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:05.351281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae191b-3060-4f6b-b21c-da424aec5a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:59:05.352741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae191b-3060-4f6b-b21c-da424aec5a96 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:59:05.370721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 748/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:59:07.597213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2309ab-becf-4abb-ae52-65ec84c18059 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:07.597896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2309ab-becf-4abb-ae52-65ec84c18059 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:07.597896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2309ab-becf-4abb-ae52-65ec84c18059 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:07.599346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 737/1485] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:59:07.604792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6903d12-d162-4f0d-9ad4-715e6d950fa1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:07.605221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6903d12-d162-4f0d-9ad4-715e6d950fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:07.605351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6903d12-d162-4f0d-9ad4-715e6d950fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:07.671759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b6903d12-d162-4f0d-9ad4-715e6d950fa1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:59:07.672938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b6903d12-d162-4f0d-9ad4-715e6d950fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:07.769146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 749/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:59:08.061634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2309ab-becf-4abb-ae52-65ec84c18059 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:59:08.063172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0c2309ab-becf-4abb-ae52-65ec84c18059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:08.164575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2309ab-becf-4abb-ae52-65ec84c18059 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:59:08.239398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:08.241191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a0eb3-b467-4564-bbd5-175bb589b72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:59:08.241827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0eb3-b467-4564-bbd5-175bb589b72f service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:08.242311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0eb3-b467-4564-bbd5-175bb589b72f service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:08.242807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0eb3-b467-4564-bbd5-175bb589b72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:08.244795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d34a0eb3-b467-4564-bbd5-175bb589b72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:08.355247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a0eb3-b467-4564-bbd5-175bb589b72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:59:08.356639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a0eb3-b467-4564-bbd5-175bb589b72f service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:59:08.374581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 738/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:59:10.625165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef78ce-7f57-4772-834f-d4034816863c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:10.625701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef78ce-7f57-4772-834f-d4034816863c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:10.625701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef78ce-7f57-4772-834f-d4034816863c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:10.627083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 750/1488] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:59:10.633204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b394da-6890-47f1-8f61-bd486f4ce075 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:10.633888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b394da-6890-47f1-8f61-bd486f4ce075 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:10.634113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b394da-6890-47f1-8f61-bd486f4ce075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:10.700034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a6b394da-6890-47f1-8f61-bd486f4ce075 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:59:10.701354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a6b394da-6890-47f1-8f61-bd486f4ce075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:10.753588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:10.798299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 739/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:59:11.095458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef78ce-7f57-4772-834f-d4034816863c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:59:11.097044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-53ef78ce-7f57-4772-834f-d4034816863c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:11.141423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:11.220763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef78ce-7f57-4772-834f-d4034816863c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:59:11.301827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed61e3-08ac-4799-81ac-8c782efa3916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:59:11.302465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed61e3-08ac-4799-81ac-8c782efa3916 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:11.302744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed61e3-08ac-4799-81ac-8c782efa3916 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:11.303021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed61e3-08ac-4799-81ac-8c782efa3916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:11.304698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eeed61e3-08ac-4799-81ac-8c782efa3916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:11.400181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:11.431526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed61e3-08ac-4799-81ac-8c782efa3916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:59:11.432998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed61e3-08ac-4799-81ac-8c782efa3916 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:59:11.451612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 751/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:59:13.931225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5065-f10b-44ff-b39f-5c8887e2d88e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:13.931813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5065-f10b-44ff-b39f-5c8887e2d88e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:13.931813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5065-f10b-44ff-b39f-5c8887e2d88e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:13.933138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 740/1491] 10.222.0.52 () {54 vars in 927 bytes} [Thu Oct 6 10:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 10:59:13.938518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94b64-6471-4023-bf6d-aa8f5aebc83e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:13.938914 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94b64-6471-4023-bf6d-aa8f5aebc83e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:13.939072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94b64-6471-4023-bf6d-aa8f5aebc83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:14.006026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5aa94b64-6471-4023-bf6d-aa8f5aebc83e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 10:59:14.007219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5aa94b64-6471-4023-bf6d-aa8f5aebc83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:14.105130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 752/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Oct 6 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 10:59:14.396329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5065-f10b-44ff-b39f-5c8887e2d88e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 10:59:14.397843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-53bf5065-f10b-44ff-b39f-5c8887e2d88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:14.504277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5065-f10b-44ff-b39f-5c8887e2d88e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 10:59:14.577486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c13453-bd0b-4cac-ac51-3dfcd27fe7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 10:59:14.578020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c13453-bd0b-4cac-ac51-3dfcd27fe7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 10:59:14.578300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c13453-bd0b-4cac-ac51-3dfcd27fe7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 10:59:14.578602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c13453-bd0b-4cac-ac51-3dfcd27fe7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 10:59:14.580235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-01c13453-bd0b-4cac-ac51-3dfcd27fe7c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 10:59:14.589175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:14.690701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c13453-bd0b-4cac-ac51-3dfcd27fe7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 10:59:14.692842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c13453-bd0b-4cac-ac51-3dfcd27fe7c7 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 10:59:14.710884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 741/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 10:59:24.103562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=82 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 10:59:24.708724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=50 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:27.972479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:27.973094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:27.973094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:28.057492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:00:28.058718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:28.154051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 753/1494] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:00:28.162599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbca05-8c0c-4b67-9b3f-5107d56045ba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:28.163089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbca05-8c0c-4b67-9b3f-5107d56045ba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:28.163194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbca05-8c0c-4b67-9b3f-5107d56045ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:28.237933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1dfbca05-8c0c-4b67-9b3f-5107d56045ba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:00:28.239154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1dfbca05-8c0c-4b67-9b3f-5107d56045ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:28.335321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 742/1495] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:00:28.340078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:28.341269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:28.353336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8228e-c6dd-4a44-86db-03dae2a7b475 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:28.415378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:28.449333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:28.449939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:28.450219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:28.450485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:28.451117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:00:28.451414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:28.452458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:28.452458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 11:00:28.452458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 11:00:28.453335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:28.464521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 754/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:00:28.473465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbca05-8c0c-4b67-9b3f-5107d56045ba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:28.482310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbca05-8c0c-4b67-9b3f-5107d56045ba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:28.545400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:28.578853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:28.580141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:28.580504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:28.580788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:28.581421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:28.583194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:28.583194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Oct 06 11:00:28.583194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: Oct 06 11:00:28.584429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:28.641205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 743/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:00:28 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:00:28.646781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:28.648352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:28.683114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da9993-2671-44a3-a703-7e0a8a24f927 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:28.780686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:28.781285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:28.781561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:28.781829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:28.783309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:28.784497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:28.784497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 11:00:28.784497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 11:00:28.785693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:28.826246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 755/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:00:28 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:00:28.831049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:28.832578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:28.894475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7e83c-7f2e-4c80-9cf6-e579d19ed1cc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:28.976790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:28.988185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:28.988781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:28.989056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:28.989351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:28.989982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:28.990974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:28.990974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Oct 06 11:00:28.990974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: Oct 06 11:00:28.991875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:29.004409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 744/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:00:29.008436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:29.009927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:29.021704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c864686-d801-47d5-ba8d-3e31e05d0acd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:29.046464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:29.115711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d362bf-a2e1-483f-8bee-d50a0a347964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:29.116285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d362bf-a2e1-483f-8bee-d50a0a347964 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:29.116565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d362bf-a2e1-483f-8bee-d50a0a347964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:29.116837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d362bf-a2e1-483f-8bee-d50a0a347964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:29.117475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d362bf-a2e1-483f-8bee-d50a0a347964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:29.118480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-99d362bf-a2e1-483f-8bee-d50a0a347964 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:29.118480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 11:00:29.118480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 11:00:29.119363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d362bf-a2e1-483f-8bee-d50a0a347964 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:29.132138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 756/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:00:29.136357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:29.137878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:29.171972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61847e71-9861-43fc-8e41-d24cc53cb4cb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:29.267163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154e6bb-4ebd-4a67-ae0e-a5f8a1ee5cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:29.267694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154e6bb-4ebd-4a67-ae0e-a5f8a1ee5cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:29.268003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154e6bb-4ebd-4a67-ae0e-a5f8a1ee5cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:29.268275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154e6bb-4ebd-4a67-ae0e-a5f8a1ee5cdc admin admin] PATH_INFO: `/v3/projects/1d8df8364384478f9b4ab5f095ad896d/users/b5a41a7fc37a4228b01f7190e9a7cbd4/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:29.290424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154e6bb-4ebd-4a67-ae0e-a5f8a1ee5cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d8df8364384478f9b4ab5f095ad896d, user_id=b5a41a7fc37a4228b01f7190e9a7cbd4, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:29.291434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-5154e6bb-4ebd-4a67-ae0e-a5f8a1ee5cdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:29.291434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Oct 06 11:00:29.291434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: Oct 06 11:00:29.293411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154e6bb-4ebd-4a67-ae0e-a5f8a1ee5cdc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:29.319326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 745/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:00:29 2022] PUT /identity/v3/projects/1d8df8364384478f9b4ab5f095ad896d/users/b5a41a7fc37a4228b01f7190e9a7cbd4/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:00:29.331746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c872-93a3-4597-8f9b-2cc67d83ce06 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:29.332070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c872-93a3-4597-8f9b-2cc67d83ce06 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:29.332344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c872-93a3-4597-8f9b-2cc67d83ce06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:29.392627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4db0c872-93a3-4597-8f9b-2cc67d83ce06 None None] MFA Rules not processed for user `b5a41a7fc37a4228b01f7190e9a7cbd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:00:29.393859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4db0c872-93a3-4597-8f9b-2cc67d83ce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:29.470352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 757/1502] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:00:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:00:29.483113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e783c2-9204-4031-be26-66182b55d969 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:29.483399 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e783c2-9204-4031-be26-66182b55d969 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:29.483668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e783c2-9204-4031-be26-66182b55d969 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:29.485026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 746/1503] 10.222.0.52 () {54 vars in 963 bytes} [Thu Oct 6 11:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:00:29.490297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a271a-0f36-4989-b5af-1fcf4f3cca0b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:29.490580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a271a-0f36-4989-b5af-1fcf4f3cca0b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:29.490852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a271a-0f36-4989-b5af-1fcf4f3cca0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:29.522303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:29.567057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f58a271a-0f36-4989-b5af-1fcf4f3cca0b None None] MFA Rules not processed for user `6ecccd27dfa04cf985c9fd632ce2fe60`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:00:29.568278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f58a271a-0f36-4989-b5af-1fcf4f3cca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:29.644281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 758/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Oct 6 11:00:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:00:29.652251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e783c2-9204-4031-be26-66182b55d969 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:29.653744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e4e783c2-9204-4031-be26-66182b55d969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:29.702069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:29.755977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e783c2-9204-4031-be26-66182b55d969 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:29.828567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:29.829797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:29.830083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:29.830406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:29.832088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:29.930484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:29.931852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:29.935284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:29.950459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 747/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:00:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:00:29.996815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebb737-f45e-4510-b6fe-8cc2ee3bb063 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:29.997102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebb737-f45e-4510-b6fe-8cc2ee3bb063 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:29.997378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebb737-f45e-4510-b6fe-8cc2ee3bb063 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:29.998654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 759/1506] 10.222.0.52 () {52 vars in 830 bytes} [Thu Oct 6 11:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:00:30.002966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:30.004233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:30.102092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b69e61-2e0d-46df-a728-cc437b9ac572 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:30.194991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:30.195638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:30.195816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:30.196121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] PATH_INFO: `/v3/users/b5a41a7fc37a4228b01f7190e9a7cbd4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:30.209095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5a41a7fc37a4228b01f7190e9a7cbd4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:30.210110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:30.210110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Oct 06 11:00:30.210110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: Oct 06 11:00:30.211331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:30.220864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:30.276979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5a41a7fc37a4228b01f7190e9a7cbd4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:30.299884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5a41a7fc37a4228b01f7190e9a7cbd4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:30.309181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5a41a7fc37a4228b01f7190e9a7cbd4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:30.309793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5a41a7fc37a4228b01f7190e9a7cbd4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:30.324305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5a41a7fc37a4228b01f7190e9a7cbd4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:30.329852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 748/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:00:30 2022] DELETE /identity/v3/users/b5a41a7fc37a4228b01f7190e9a7cbd4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:00:30.334179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:30.334600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:30.334705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:30.391513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:00:30.392930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:30.467914 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:30.481510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 760/1508] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:00:30.492220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:30.493414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:30.594958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e860c-9314-48b2-bbd8-161f5d5b9b1c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:30.651633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:30.673703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8903967-fd3b-4122-a650-0e8929c46913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:30.674312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903967-fd3b-4122-a650-0e8929c46913 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:30.674585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903967-fd3b-4122-a650-0e8929c46913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:30.674855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903967-fd3b-4122-a650-0e8929c46913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:30.676467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8903967-fd3b-4122-a650-0e8929c46913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:30.791172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8903967-fd3b-4122-a650-0e8929c46913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:30.792731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8903967-fd3b-4122-a650-0e8929c46913 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:30.811626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 749/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:00:31.156661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:00:31.158643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:00:31.170449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14627271-86dd-42bf-98b8-7b1b71d895c3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:00:31.268976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:00:31.269610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:00:31.269880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:00:31.270154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] PATH_INFO: `/v3/projects/1d8df8364384478f9b4ab5f095ad896d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:00:31.281001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d8df8364384478f9b4ab5f095ad896d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:00:31.282030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:00:31.282030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 11:00:31.282030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 11:00:31.283236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:00:31.341144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d8df8364384478f9b4ab5f095ad896d', 'request_id': 'req-63cdc119-f0bf-40e7-a0c8-698350b65fc7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:31.347307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d8df8364384478f9b4ab5f095ad896d', 'request_id': 'req-63cdc119-f0bf-40e7-a0c8-698350b65fc7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:31.360409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d8df8364384478f9b4ab5f095ad896d', 'request_id': 'req-63cdc119-f0bf-40e7-a0c8-698350b65fc7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:31.386908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] The token cache is being invalidate because project 1d8df8364384478f9b4ab5f095ad896d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:00:31.387199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:31.387633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-63cdc119-f0bf-40e7-a0c8-698350b65fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:00:31.389077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 761/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:00:31 2022] DELETE /identity/v3/projects/1d8df8364384478f9b4ab5f095ad896d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:00:40.809392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=62 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:00:41.384930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:02:00.718162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489dc4f-0225-4290-872c-9e342b2639d1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:00.718731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489dc4f-0225-4290-872c-9e342b2639d1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:00.718954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489dc4f-0225-4290-872c-9e342b2639d1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:00.720382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 11:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:00.725164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4dd39-72e7-4733-926d-4865f8d0e618 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:00.725588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4dd39-72e7-4733-926d-4865f8d0e618 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:00.725713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4dd39-72e7-4733-926d-4865f8d0e618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:00.806712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c8f4dd39-72e7-4733-926d-4865f8d0e618 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:00.807962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c8f4dd39-72e7-4733-926d-4865f8d0e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:00.910929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:01.069689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489dc4f-0225-4290-872c-9e342b2639d1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:01.071804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d489dc4f-0225-4290-872c-9e342b2639d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:01.197441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489dc4f-0225-4290-872c-9e342b2639d1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:01.203147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877341-a303-4a26-bde6-c70f2aab93b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:01.203458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877341-a303-4a26-bde6-c70f2aab93b2 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:01.203733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877341-a303-4a26-bde6-c70f2aab93b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:01.204036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877341-a303-4a26-bde6-c70f2aab93b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:01.212009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877341-a303-4a26-bde6-c70f2aab93b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:01.213576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877341-a303-4a26-bde6-c70f2aab93b2 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:01.223378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 751/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:02:02.299696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60b4a3-e13d-472a-b987-32e5aa116c4f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:02.300307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60b4a3-e13d-472a-b987-32e5aa116c4f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:02.300307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60b4a3-e13d-472a-b987-32e5aa116c4f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:02.301646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 11:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:02.307832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2693eb4-f494-441c-9874-9529bb5de014 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:02.308323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2693eb4-f494-441c-9874-9529bb5de014 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:02.308435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2693eb4-f494-441c-9874-9529bb5de014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:02.328211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a2693eb4-f494-441c-9874-9529bb5de014 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:02.329405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a2693eb4-f494-441c-9874-9529bb5de014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:02.337091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 11:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:02.490728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60b4a3-e13d-472a-b987-32e5aa116c4f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:02.505795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60b4a3-e13d-472a-b987-32e5aa116c4f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:02.511266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e56a2-6afe-4a3f-b75a-8c83e0b6410d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:02.511889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e56a2-6afe-4a3f-b75a-8c83e0b6410d service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:02.512196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e56a2-6afe-4a3f-b75a-8c83e0b6410d service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:02.512473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e56a2-6afe-4a3f-b75a-8c83e0b6410d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:02.520685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e56a2-6afe-4a3f-b75a-8c83e0b6410d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:02.522270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e56a2-6afe-4a3f-b75a-8c83e0b6410d service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:02.532083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 764/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:02:02.568727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc704103-a38a-4cdc-8b5b-931c319ff54c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:02.569021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc704103-a38a-4cdc-8b5b-931c319ff54c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:02.569302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc704103-a38a-4cdc-8b5b-931c319ff54c None None] PATH_INFO: `/v3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:02.570605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 753/1517] 10.222.0.55 () {54 vars in 925 bytes} [Thu Oct 6 11:02:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:02:02.577294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381798ab-cb24-43e8-9bec-792a49f6b28b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:02.578434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381798ab-cb24-43e8-9bec-792a49f6b28b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:02.578710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381798ab-cb24-43e8-9bec-792a49f6b28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:02.649500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-381798ab-cb24-43e8-9bec-792a49f6b28b None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:02.650739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-381798ab-cb24-43e8-9bec-792a49f6b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:02.669650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:02:02.726886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 765/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Oct 6 11:02:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:11.598876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1566-0981-4761-8489-660ada39f9da None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:11.599354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1566-0981-4761-8489-660ada39f9da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:11.599457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1566-0981-4761-8489-660ada39f9da None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:11.600893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 754/1519] 10.222.0.52 () {54 vars in 961 bytes} [Thu Oct 6 11:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:11.605956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f607-f58c-4253-b52d-99d6dd71e0d6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:11.606401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f607-f58c-4253-b52d-99d6dd71e0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:11.606581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f607-f58c-4253-b52d-99d6dd71e0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:11.626708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8b61f607-f58c-4253-b52d-99d6dd71e0d6 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:11.627970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8b61f607-f58c-4253-b52d-99d6dd71e0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:11.637747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 766/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Oct 6 11:02:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:11.645917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1566-0981-4761-8489-660ada39f9da None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:11.659972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1566-0981-4761-8489-660ada39f9da None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:11.664850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18b132-bb87-45ef-aab6-71375d7ec6f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:11.665439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18b132-bb87-45ef-aab6-71375d7ec6f2 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:11.665716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18b132-bb87-45ef-aab6-71375d7ec6f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:11.665988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18b132-bb87-45ef-aab6-71375d7ec6f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:11.674827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18b132-bb87-45ef-aab6-71375d7ec6f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:11.676413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18b132-bb87-45ef-aab6-71375d7ec6f2 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:11.686134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 755/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:02:14.025824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0148-f223-4676-8c2c-423b955a17d2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:14.026403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0148-f223-4676-8c2c-423b955a17d2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:14.026403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0148-f223-4676-8c2c-423b955a17d2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:14.027729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 11:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:14.033787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ad565-6fdc-4058-ae2e-e0b905bb6b82 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:14.034206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ad565-6fdc-4058-ae2e-e0b905bb6b82 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:14.034367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ad565-6fdc-4058-ae2e-e0b905bb6b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:14.054245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-581ad565-6fdc-4058-ae2e-e0b905bb6b82 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:14.055473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-581ad565-6fdc-4058-ae2e-e0b905bb6b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:14.064198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 11:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:14.215564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf0148-f223-4676-8c2c-423b955a17d2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:14.229907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf0148-f223-4676-8c2c-423b955a17d2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:14.235750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f004703-ec01-4231-8d41-49f63299f700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:14.236388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f004703-ec01-4231-8d41-49f63299f700 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:14.236670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f004703-ec01-4231-8d41-49f63299f700 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:14.236940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f004703-ec01-4231-8d41-49f63299f700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:14.244760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f004703-ec01-4231-8d41-49f63299f700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:14.246324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f004703-ec01-4231-8d41-49f63299f700 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:14.255974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 768/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:02:15.327612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa29db-ccd3-4ece-93f4-16f13c7fdb1e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:15.328300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa29db-ccd3-4ece-93f4-16f13c7fdb1e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:15.328300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa29db-ccd3-4ece-93f4-16f13c7fdb1e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:15.329605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 11:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:15.335867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239d3a-4a33-4a31-adaf-bfb75543122b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:15.336313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239d3a-4a33-4a31-adaf-bfb75543122b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:15.336418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239d3a-4a33-4a31-adaf-bfb75543122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:15.355808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cf239d3a-4a33-4a31-adaf-bfb75543122b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:15.357040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf239d3a-4a33-4a31-adaf-bfb75543122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:15.365528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:15.517426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa29db-ccd3-4ece-93f4-16f13c7fdb1e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:15.532329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa29db-ccd3-4ece-93f4-16f13c7fdb1e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:15.537778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7e132-f5fb-4bd1-a499-6722e7620f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:15.538377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7e132-f5fb-4bd1-a499-6722e7620f76 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:15.538651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7e132-f5fb-4bd1-a499-6722e7620f76 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:15.538925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7e132-f5fb-4bd1-a499-6722e7620f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:15.547150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7e132-f5fb-4bd1-a499-6722e7620f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:15.548808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7e132-f5fb-4bd1-a499-6722e7620f76 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:15.559200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 758/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:02:19.962813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684867c5-91d4-4078-9e2b-c0952391b619 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:19.963400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684867c5-91d4-4078-9e2b-c0952391b619 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:19.963400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684867c5-91d4-4078-9e2b-c0952391b619 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:19.964733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 11:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:19.970803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7793bd-a2b1-40e7-b93d-7db126542d3e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:19.971211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7793bd-a2b1-40e7-b93d-7db126542d3e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:19.971358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7793bd-a2b1-40e7-b93d-7db126542d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:19.991131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1e7793bd-a2b1-40e7-b93d-7db126542d3e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:19.992367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e7793bd-a2b1-40e7-b93d-7db126542d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:20.000490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:20.291551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684867c5-91d4-4078-9e2b-c0952391b619 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:20.293242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-684867c5-91d4-4078-9e2b-c0952391b619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:20.371349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684867c5-91d4-4078-9e2b-c0952391b619 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:20.374945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b492ef-7a0d-4c8c-8107-0c369a2f3897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:20.375623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b492ef-7a0d-4c8c-8107-0c369a2f3897 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:20.375935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b492ef-7a0d-4c8c-8107-0c369a2f3897 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:20.376214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b492ef-7a0d-4c8c-8107-0c369a2f3897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:20.385170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b492ef-7a0d-4c8c-8107-0c369a2f3897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:20.386643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b492ef-7a0d-4c8c-8107-0c369a2f3897 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:20.396859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 771/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:02:22.542233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df0e63-ce86-4a0a-a088-a9431a9a07d0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:22.542822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df0e63-ce86-4a0a-a088-a9431a9a07d0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:22.542822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df0e63-ce86-4a0a-a088-a9431a9a07d0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:22.544175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 11:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:22.550188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc34cb2-402b-4f41-80c0-7172f62b6ba0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:22.550607 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc34cb2-402b-4f41-80c0-7172f62b6ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:22.550740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc34cb2-402b-4f41-80c0-7172f62b6ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:22.570537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6bc34cb2-402b-4f41-80c0-7172f62b6ba0 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:22.571725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6bc34cb2-402b-4f41-80c0-7172f62b6ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:22.579553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:22.870420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df0e63-ce86-4a0a-a088-a9431a9a07d0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:22.881115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df0e63-ce86-4a0a-a088-a9431a9a07d0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:22.884553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4206a76-d087-47ac-a14e-7098896983e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:22.885139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206a76-d087-47ac-a14e-7098896983e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:22.885415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206a76-d087-47ac-a14e-7098896983e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:22.885688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206a76-d087-47ac-a14e-7098896983e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:22.894310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4206a76-d087-47ac-a14e-7098896983e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:22.895711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4206a76-d087-47ac-a14e-7098896983e2 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:22.905114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 761/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:02:26.609634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431014-7569-482a-9895-449c540b759e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:26.610209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431014-7569-482a-9895-449c540b759e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:26.610209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431014-7569-482a-9895-449c540b759e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:26.611545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 11:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:26.616826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fb479-4ab2-4a4a-8cac-a26ed90fed33 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:26.617219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fb479-4ab2-4a4a-8cac-a26ed90fed33 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:26.617370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fb479-4ab2-4a4a-8cac-a26ed90fed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:26.637884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-612fb479-4ab2-4a4a-8cac-a26ed90fed33 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:26.638988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-612fb479-4ab2-4a4a-8cac-a26ed90fed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:26.647355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 11:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:26.948922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba431014-7569-482a-9895-449c540b759e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:26.950627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ba431014-7569-482a-9895-449c540b759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:27.019437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:02:27.034785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba431014-7569-482a-9895-449c540b759e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:27.038391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c44b53-da01-494c-be6b-8e0aa214c6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:27.039002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44b53-da01-494c-be6b-8e0aa214c6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:27.039348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44b53-da01-494c-be6b-8e0aa214c6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:27.039556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44b53-da01-494c-be6b-8e0aa214c6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:27.049159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c44b53-da01-494c-be6b-8e0aa214c6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:27.050584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c44b53-da01-494c-be6b-8e0aa214c6a4 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:27.059945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 774/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:02:29.201135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8910ab0-88bb-4e6d-b514-cc5e5668f3f6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:29.201665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8910ab0-88bb-4e6d-b514-cc5e5668f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:29.201665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8910ab0-88bb-4e6d-b514-cc5e5668f3f6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:29.204035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 11:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:02:29.209145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c21113-7f04-4347-9227-471ea8951cd0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:29.209568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c21113-7f04-4347-9227-471ea8951cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:29.209702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c21113-7f04-4347-9227-471ea8951cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:29.229556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-65c21113-7f04-4347-9227-471ea8951cd0 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:02:29.230759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-65c21113-7f04-4347-9227-471ea8951cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:29.238917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 11:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:02:29.552588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8910ab0-88bb-4e6d-b514-cc5e5668f3f6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:02:29.554289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a8910ab0-88bb-4e6d-b514-cc5e5668f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:02:29.568125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8910ab0-88bb-4e6d-b514-cc5e5668f3f6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:02:29.571692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79cfc9-049b-4a53-a999-c2aa40413a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:02:29.572310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cfc9-049b-4a53-a999-c2aa40413a5e service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:02:29.572590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cfc9-049b-4a53-a999-c2aa40413a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:02:29.572862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cfc9-049b-4a53-a999-c2aa40413a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:02:29.581924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79cfc9-049b-4a53-a999-c2aa40413a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:02:29.583326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79cfc9-049b-4a53-a999-c2aa40413a5e service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:02:29.592611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 764/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:02:39.224833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=11 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:02:39.588496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:53.545955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ab0f3-5031-4798-af33-7c4ecf3d59d8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:53.546804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ab0f3-5031-4798-af33-7c4ecf3d59d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:53.546804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ab0f3-5031-4798-af33-7c4ecf3d59d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:53.576845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bb4ab0f3-5031-4798-af33-7c4ecf3d59d8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:53.579381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bb4ab0f3-5031-4798-af33-7c4ecf3d59d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:53.588680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 776/1540] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:53.597366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69026c4-b4c1-4b4c-b816-27808b1d4728 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:53.597946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69026c4-b4c1-4b4c-b816-27808b1d4728 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:53.597946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69026c4-b4c1-4b4c-b816-27808b1d4728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:53.622585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d69026c4-b4c1-4b4c-b816-27808b1d4728 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:53.623805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d69026c4-b4c1-4b4c-b816-27808b1d4728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:53.637603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 765/1541] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:53.643425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ab0f3-5031-4798-af33-7c4ecf3d59d8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:53.657482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ab0f3-5031-4798-af33-7c4ecf3d59d8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:53.672039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:53.673099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:53.673383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:53.673658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:53.674326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:53.674631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:53.676315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:53.694582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 777/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:53.704357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69026c4-b4c1-4b4c-b816-27808b1d4728 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:53.717976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69026c4-b4c1-4b4c-b816-27808b1d4728 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:53.726915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:53.727975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:53.728490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:53.729006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:53.729887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:53.732103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:53.807333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 766/1543] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:53 2022] POST /identity/v3/projects => generated 353 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:53.810059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:53.828902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460b2a-6adc-4ee1-be8f-7066fc08f65b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:53.921950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:53.922649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:53.922854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:53.923132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:53.923785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:53.925989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:53.962274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 778/1544] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:53 2022] POST /identity/v3/users => generated 277 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:53.968861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:53.980947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded9b70-a6c6-4386-afec-d80f175f0542 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:53.988052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:53.988687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:53.988970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:53.989240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:53.989888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:53.991566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.009083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.010173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 767/1545] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:54.022131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7520-33a3-47b9-b51a-3c45e917da53 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.028209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5e702-2fa9-4347-842e-414e816a65c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.028826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5e702-2fa9-4347-842e-414e816a65c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.029106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5e702-2fa9-4347-842e-414e816a65c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.029379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5e702-2fa9-4347-842e-414e816a65c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.030312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5e702-2fa9-4347-842e-414e816a65c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.032365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5e702-2fa9-4347-842e-414e816a65c9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.048520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 779/1546] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:54.053035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.068521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb8802-a974-4316-8a71-66bf8a193ad9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.075993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83bd45-9f8b-4099-9e8c-ca469d6303dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.076688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bd45-9f8b-4099-9e8c-ca469d6303dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.077416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bd45-9f8b-4099-9e8c-ca469d6303dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.077854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bd45-9f8b-4099-9e8c-ca469d6303dc admin admin] PATH_INFO: `/v3/projects/2c108c50400744fa8e39a9f770625031/users/baf7d8e8af204cbd830a79f54fb680ac/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.102332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83bd45-9f8b-4099-9e8c-ca469d6303dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c108c50400744fa8e39a9f770625031, user_id=baf7d8e8af204cbd830a79f54fb680ac, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.105376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83bd45-9f8b-4099-9e8c-ca469d6303dc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.137423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 768/1547] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:54 2022] PUT /identity/v3/projects/2c108c50400744fa8e39a9f770625031/users/baf7d8e8af204cbd830a79f54fb680ac/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:54.155199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f5c5-3a17-41cf-93be-002538fd18d4 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.155591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f5c5-3a17-41cf-93be-002538fd18d4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.155877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f5c5-3a17-41cf-93be-002538fd18d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.184957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8d2c-7b7a-474a-a9f7-2e473343eec8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.185422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8d2c-7b7a-474a-a9f7-2e473343eec8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.185705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8d2c-7b7a-474a-a9f7-2e473343eec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.229495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-41bc8d2c-7b7a-474a-a9f7-2e473343eec8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.229736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2312f5c5-3a17-41cf-93be-002538fd18d4 None None] MFA Rules not processed for user `baf7d8e8af204cbd830a79f54fb680ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.230913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2312f5c5-3a17-41cf-93be-002538fd18d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.231119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-41bc8d2c-7b7a-474a-a9f7-2e473343eec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.342164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 769/1548] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.348538 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485e75-82d3-4ed3-bb0d-4f2836bb1280 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.348819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485e75-82d3-4ed3-bb0d-4f2836bb1280 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.349093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485e75-82d3-4ed3-bb0d-4f2836bb1280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.366866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 780/1549] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.368353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30ab3d-e908-494c-859f-d3e90f999d6d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.368644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30ab3d-e908-494c-859f-d3e90f999d6d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.368922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30ab3d-e908-494c-859f-d3e90f999d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.375568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3c485e75-82d3-4ed3-bb0d-4f2836bb1280 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.376824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3c485e75-82d3-4ed3-bb0d-4f2836bb1280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.386069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 770/1550] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.387337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f1b25-33fe-44be-86e9-b3d0f891e0b8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.387620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f1b25-33fe-44be-86e9-b3d0f891e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.387920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f1b25-33fe-44be-86e9-b3d0f891e0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.398857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0b30ab3d-e908-494c-859f-d3e90f999d6d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.400739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0b30ab3d-e908-494c-859f-d3e90f999d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.409759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 781/1551] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.411215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26013ff3-b092-43e5-9c8e-a9bcd338ace1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.411629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26013ff3-b092-43e5-9c8e-a9bcd338ace1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.411966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26013ff3-b092-43e5-9c8e-a9bcd338ace1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.416910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-547f1b25-33fe-44be-86e9-b3d0f891e0b8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.418116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-547f1b25-33fe-44be-86e9-b3d0f891e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.426781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 771/1552] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.428049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22921ade-2fdd-42dc-a8bd-65bd8492b0b2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.428332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22921ade-2fdd-42dc-a8bd-65bd8492b0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.428607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22921ade-2fdd-42dc-a8bd-65bd8492b0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.437804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-26013ff3-b092-43e5-9c8e-a9bcd338ace1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.439038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-26013ff3-b092-43e5-9c8e-a9bcd338ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.448890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 782/1553] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.449999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26013ff3-b092-43e5-9c8e-a9bcd338ace1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.453665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-22921ade-2fdd-42dc-a8bd-65bd8492b0b2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.454860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-22921ade-2fdd-42dc-a8bd-65bd8492b0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.459724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26013ff3-b092-43e5-9c8e-a9bcd338ace1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.462686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 772/1554] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.463892 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5743-a06e-43c9-aeba-fe4e946a3e4e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.464181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5743-a06e-43c9-aeba-fe4e946a3e4e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.464460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5743-a06e-43c9-aeba-fe4e946a3e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.464656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eab5c7-c49c-481d-a3ca-b508895eb596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.465235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eab5c7-c49c-481d-a3ca-b508895eb596 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.465512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eab5c7-c49c-481d-a3ca-b508895eb596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.465780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eab5c7-c49c-481d-a3ca-b508895eb596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.466398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eab5c7-c49c-481d-a3ca-b508895eb596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.467409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-16eab5c7-c49c-481d-a3ca-b508895eb596 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:03:54.467409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 11:03:54.467409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 11:03:54.469153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eab5c7-c49c-481d-a3ca-b508895eb596 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.485818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-690c5743-a06e-43c9-aeba-fe4e946a3e4e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.486990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-690c5743-a06e-43c9-aeba-fe4e946a3e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.495801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 773/1555] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.497037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd30987-bc2c-40e7-b013-7ad8432f6226 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.497325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd30987-bc2c-40e7-b013-7ad8432f6226 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.497639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 783/1556] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:54.498964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b698a87-0e08-48dc-90b4-ea087453b6a6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.499300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b698a87-0e08-48dc-90b4-ea087453b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.499665 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b698a87-0e08-48dc-90b4-ea087453b6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.500664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd30987-bc2c-40e7-b013-7ad8432f6226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.519601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4b698a87-0e08-48dc-90b4-ea087453b6a6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.519941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9fd30987-bc2c-40e7-b013-7ad8432f6226 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.520848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4b698a87-0e08-48dc-90b4-ea087453b6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.521390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9fd30987-bc2c-40e7-b013-7ad8432f6226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.529589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:54.583051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 784/1557] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.588376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 774/1558] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.588549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607b7a3-1074-43c9-865a-3580c0e1069b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.588549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607b7a3-1074-43c9-865a-3580c0e1069b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.588549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607b7a3-1074-43c9-865a-3580c0e1069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.588824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b011cde-1a6d-41be-8b09-204674a0b2a8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.588824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b011cde-1a6d-41be-8b09-204674a0b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.588824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b011cde-1a6d-41be-8b09-204674a0b2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.595195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:54.606597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f607b7a3-1074-43c9-865a-3580c0e1069b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.607797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f607b7a3-1074-43c9-865a-3580c0e1069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.609810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4b011cde-1a6d-41be-8b09-204674a0b2a8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.610975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b011cde-1a6d-41be-8b09-204674a0b2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.616941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 785/1559] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.618115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9a78f-d125-43c5-a45b-4f38f5ce1820 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.618399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9a78f-d125-43c5-a45b-4f38f5ce1820 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.618673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9a78f-d125-43c5-a45b-4f38f5ce1820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.619373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 775/1560] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.620005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b011cde-1a6d-41be-8b09-204674a0b2a8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.630835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b011cde-1a6d-41be-8b09-204674a0b2a8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.635726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.636435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.636721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.636996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.637639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:54.637939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.638945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:03:54.638945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Oct 06 11:03:54.638945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: Oct 06 11:03:54.639988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c402b-9137-485f-affa-3268d3771def admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.640484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b6c9a78f-d125-43c5-a45b-4f38f5ce1820 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.641975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b6c9a78f-d125-43c5-a45b-4f38f5ce1820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.650316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 786/1561] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.651065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9a78f-d125-43c5-a45b-4f38f5ce1820 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.659389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 776/1562] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:54.659389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ea1e-e515-4fb6-bba2-3247d2609a2c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.659389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ea1e-e515-4fb6-bba2-3247d2609a2c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.659389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ea1e-e515-4fb6-bba2-3247d2609a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.660901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9a78f-d125-43c5-a45b-4f38f5ce1820 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.665387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.666002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.666276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.666838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.667462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.669426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.673682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4587ea1e-e515-4fb6-bba2-3247d2609a2c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.674864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4587ea1e-e515-4fb6-bba2-3247d2609a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.683396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 777/1563] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.688974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587ea1e-e515-4fb6-bba2-3247d2609a2c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.696308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 787/1564] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/users => generated 285 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:54.696308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.700478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587ea1e-e515-4fb6-bba2-3247d2609a2c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.704751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28c3fc-a587-487c-ab44-944580a87fd9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.705896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.706517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.706797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.707094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.707732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:54.708059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.709643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a00e58-9452-4a0b-aff7-6f426da590c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.709790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.710395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a00e58-9452-4a0b-aff7-6f426da590c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.710681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a00e58-9452-4a0b-aff7-6f426da590c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.711127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a00e58-9452-4a0b-aff7-6f426da590c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.711617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a00e58-9452-4a0b-aff7-6f426da590c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:54.711951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a00e58-9452-4a0b-aff7-6f426da590c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.713534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a00e58-9452-4a0b-aff7-6f426da590c1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.722304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 778/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:54.723307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.726791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 788/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:54.728090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135ac0-9698-43b0-a4d8-226f0f2b2159 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.728434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135ac0-9698-43b0-a4d8-226f0f2b2159 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.728772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135ac0-9698-43b0-a4d8-226f0f2b2159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.735549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36fbae-1d4d-4202-8fe1-e15b114d9de5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.740383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.741013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.741391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.741670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.742302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:54.742598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.744193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.750986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-db135ac0-9698-43b0-a4d8-226f0f2b2159 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:54.752320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-db135ac0-9698-43b0-a4d8-226f0f2b2159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:54.756405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 779/1567] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:54.757224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.763535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 789/1568] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:54.763535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db135ac0-9698-43b0-a4d8-226f0f2b2159 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.767318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c330b-a32f-4455-9f6f-54895e5bd0fe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.772231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.773075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.773352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.773622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.774252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:54.774550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.775013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db135ac0-9698-43b0-a4d8-226f0f2b2159 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.776160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.779548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.780184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.781054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.781492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.782116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.784391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.788447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 780/1569] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:54.789067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.798777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6604845-52a7-470d-bf23-a0cae76a218a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.803314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.803922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.804201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.804475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.805104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:54.805403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.806968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.818978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 790/1570] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:54.819806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.826787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 781/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:54.828261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.838010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c7e1-8c65-4ad8-8654-f074850144f5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.883493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154893f7-705e-4334-adf6-96656e47d4f8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.888191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.888795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.889074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.889374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.890667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.892387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.894755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.895690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.896000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.896270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.896897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.899481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.903156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 791/1572] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:54.904113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.915508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66104cf4-c706-4d9a-bc9d-4e277501d27e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:54.923090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:54.923881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:54.924162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:54.924433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:54.925317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:54.927214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:54.930642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 782/1573] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:54.931632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.956611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 792/1574] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:54.958316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:54.977680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:54.995093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c9fba-c75f-4518-a4e1-8f4f1e7e5fe8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.007099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:55.023931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c999f9-d1b4-41e1-ac16-e5e5c9f8d021 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.028588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.029189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.029471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.029747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.030395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.033262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.055056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.055661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.056077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.056356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.056997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:03:55.057302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.058886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.061309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 793/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.062949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.082135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 783/1576] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:03:55.082135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.093041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f165f-0238-4fba-bf8f-5260a7a777e7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.127413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb671b-c485-42f5-b31f-ca6e1f854f2e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.133230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.133883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.134427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.134705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.135362 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.137291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.150113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.150719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.151000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.151271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.151920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.152922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:03:55.152922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Oct 06 11:03:55.152922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: Oct 06 11:03:55.154116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.165602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 794/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.166527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.176746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 784/1578] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.180551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.190539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef158dea-3fbe-4c1c-a31a-7f03c3056ecf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.231124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550f842-9718-4b77-9a81-9709be2f506a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.236475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.237275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.237556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.237831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.238461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.240407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.247080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.247738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.248062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.248338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] PATH_INFO: `/v3/projects/bcc4792bd74946afb4f48d18141578a9/users/db5f1d51cc8e412ab01c6d693f496357/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.264312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc4792bd74946afb4f48d18141578a9, user_id=db5f1d51cc8e412ab01c6d693f496357, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.267286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 795/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.268120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.268721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.282745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:55.286386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 785/1580] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:55 2022] PUT /identity/v3/projects/bcc4792bd74946afb4f48d18141578a9/users/db5f1d51cc8e412ab01c6d693f496357/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:55.286386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.309246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:55.338210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e005fab-9772-4dca-b4d1-8a7214f73cb3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.348082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49debfce-5e43-4279-bb6c-7beb6df07e6c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.352893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.353534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.353812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.354084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.354718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.356845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.380810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 786/1581] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.381685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.392656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8806c0-5a2d-4fd2-a8df-47a8d4068524 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.397446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.397882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.398297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.398405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.398576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.398694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.398852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.398961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.399457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.399564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.401394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.401587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.426898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 796/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.427095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 787/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.427195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.427284 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.437714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd325-280e-4425-931e-b685da69c708 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.437908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b19755-9b87-4c9b-914b-60dc48fce633 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.442369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.442516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.442990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.443102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.443271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.443408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.443547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.443686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.444226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.444376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.446141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.446288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.469994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 788/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.471319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.472392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 797/1585] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.473190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.481858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef070b14-203e-45e6-812f-07a37944ef50 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.536927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbbb8d-57b0-4fa1-9065-c6f5414ff1d1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.539162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.539766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.540071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.540342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.540981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.541497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.542078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.542353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.542562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.542673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.543231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.545156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.554328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 789/1586] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.556311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.568951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59908216-d036-4823-b0d1-d37526eeb40c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.573605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 798/1587] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.573605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.575602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.576375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.576567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.576811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.578613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.579659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.581952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e61fde-1f54-44a8-9300-122a74959e3a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.586858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.587428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.587709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.588015 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.588790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.589733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 790/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.590372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.590600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.601306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 799/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.602160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.603122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c30b2-7e10-41ca-b96d-55ee4a8e59e5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.607774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.608413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.609043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.609425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.609975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.610512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:55.612399 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.615296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cb896-0605-4d9c-baf4-53c8f9ff9b10 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.621745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.622329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.622606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.622875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.623261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 791/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.623505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.624726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.625155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.634624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148efd63-ee0d-4c5c-90e5-2bfbeaa7b053 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.636852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 800/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.637695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.639592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.640909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.641374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.641651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.642346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.643899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.648508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3190-9acb-4aa0-a963-e569683435d1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.653023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.653600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.653878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.654137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 792/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.654254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.654846 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.654960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.657838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.664998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3679d9-42b3-47cc-9eaa-a74d02ffd8e4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.670210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.670959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.671448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.671669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.672626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.674428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.679778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 801/1593] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:55.680717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.685336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 793/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.686351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.693411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd36716-f348-46cb-b013-64ca4dd5926c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.696110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11313b-7547-4e98-a79a-d4b36c214b51 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.697930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.698514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.698789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.699059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.699692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.700694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.701299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.701429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.701594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.701872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] PATH_INFO: `/v3/projects/bcc4792bd74946afb4f48d18141578a9/users/db5f1d51cc8e412ab01c6d693f496357/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.704568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc4792bd74946afb4f48d18141578a9, user_id=db5f1d51cc8e412ab01c6d693f496357, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.707194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.711814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 802/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.712598 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.722189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 794/1596] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:55 2022] PUT /identity/v3/projects/bcc4792bd74946afb4f48d18141578a9/users/db5f1d51cc8e412ab01c6d693f496357/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:55.722802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.723123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7285-8256-47b3-8cfc-90e7f96a4274 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.748041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:55.788414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.789067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.789349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.789744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.790386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.792067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.794701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143306a2-e380-485d-a379-07512ef6bdca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.799383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.800021 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.800306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.800583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.801229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.802061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 803/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.805387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.805517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.813175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebb4be-04e3-4232-a98d-118c369c1738 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.813934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 795/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.814758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.819934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.821011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.821011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.821289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.821720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.823514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.824243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9775f-ef16-4389-a8ea-d10bec401c7d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.829656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.830249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.830533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.830808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.831462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.833056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.834454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 804/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.835385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.844995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfde62-aa88-49f0-9956-608036bc2651 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.846494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 796/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.847181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.852781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.853367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.853644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.853943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.854597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.856240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.857048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f7349-22fd-4c66-8295-68457ff42748 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.862104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6245fc-ff11-4ef6-bd32-c5e08887e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.862767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6245fc-ff11-4ef6-bd32-c5e08887e680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.863048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6245fc-ff11-4ef6-bd32-c5e08887e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.863477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6245fc-ff11-4ef6-bd32-c5e08887e680 admin admin] PATH_INFO: `/v3/projects/69b37e6e45a4446093bd3bd7b3524a67/users/617e3cd4a7f84e629d724160a6dcde61/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.866520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 805/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.867434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.877247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fae26-5752-4849-904b-cc6f76914bd8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.879418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6245fc-ff11-4ef6-bd32-c5e08887e680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b37e6e45a4446093bd3bd7b3524a67, user_id=617e3cd4a7f84e629d724160a6dcde61, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.881772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.882145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6245fc-ff11-4ef6-bd32-c5e08887e680 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.882346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.882623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.882895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.883517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.885208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:55.897008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 797/1602] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:55 2022] PUT /identity/v3/projects/69b37e6e45a4446093bd3bd7b3524a67/users/617e3cd4a7f84e629d724160a6dcde61/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:55.908078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720acd8-308a-4c36-bce9-a7e950a2664b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.908078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720acd8-308a-4c36-bce9-a7e950a2664b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.908078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720acd8-308a-4c36-bce9-a7e950a2664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.908350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 806/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:55.908350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:55.913647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b6932-09fa-432a-acf7-657a7bfbddfc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:55.928747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7720acd8-308a-4c36-bce9-a7e950a2664b None None] MFA Rules not processed for user `db5f1d51cc8e412ab01c6d693f496357`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:55.929965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7720acd8-308a-4c36-bce9-a7e950a2664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:55.971964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:55.972633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:55.972912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:55.973186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] PATH_INFO: `/v3/projects/aaa6c50978eb46ecba4c88562ede965b/users/02930aebe31e450f9fc989af369bf319/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:55.987475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa6c50978eb46ecba4c88562ede965b, user_id=02930aebe31e450f9fc989af369bf319, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:55.988620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:03:55.988620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 11:03:55.988620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 11:03:55.990551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.001818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 798/1604] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.002622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720acd8-308a-4c36-bce9-a7e950a2664b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.013072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 807/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:55 2022] PUT /identity/v3/projects/aaa6c50978eb46ecba4c88562ede965b/users/02930aebe31e450f9fc989af369bf319/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:56.013072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.013338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720acd8-308a-4c36-bce9-a7e950a2664b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.024014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:56.071438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.072133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.072429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.072712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] PATH_INFO: `/v3/projects/353c183951e4445c9e31217081964962/users/636eb68bb1314f4ba70b0e9ce039cc8d/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.076314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224d6f-08d9-4c67-ade5-442633b681d0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.076559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:56.082140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.082804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.083086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.083363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] PATH_INFO: `/v3/projects/5b886dc365684ab1811d100eb5774619/users/56835f0c8dc0416d9fa70662e40c856d/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.090199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=353c183951e4445c9e31217081964962, user_id=636eb68bb1314f4ba70b0e9ce039cc8d, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.092981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.099099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b886dc365684ab1811d100eb5774619, user_id=56835f0c8dc0416d9fa70662e40c856d, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.101799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.105760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 799/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:56 2022] PUT /identity/v3/projects/353c183951e4445c9e31217081964962/users/636eb68bb1314f4ba70b0e9ce039cc8d/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:56.106540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.114882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 808/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:56 2022] PUT /identity/v3/projects/5b886dc365684ab1811d100eb5774619/users/56835f0c8dc0416d9fa70662e40c856d/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:56.119275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.172213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efeb628-a1f0-4e26-9134-22ae1536db9c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.177031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3178a9-b521-435b-bc76-43d893449915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.177705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3178a9-b521-435b-bc76-43d893449915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.177973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3178a9-b521-435b-bc76-43d893449915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.178371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3178a9-b521-435b-bc76-43d893449915 admin admin] PATH_INFO: `/v3/projects/8c9bfd4947d045cf831f1a60bf91f389/users/1bb1115590ac45fdbcdc6d3f52aa6921/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.181028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d4de2-67d9-40c3-9ee1-cd6dcd5b37fa admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.185599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.186201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.186481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.187389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.188085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.189696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.194826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3178a9-b521-435b-bc76-43d893449915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9bfd4947d045cf831f1a60bf91f389, user_id=1bb1115590ac45fdbcdc6d3f52aa6921, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.197561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3178a9-b521-435b-bc76-43d893449915 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.199983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 809/1608] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:03:56.201038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.211599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4de514-df5f-49fa-b8f3-c55f5c6fffc6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.212945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 800/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:56 2022] PUT /identity/v3/projects/8c9bfd4947d045cf831f1a60bf91f389/users/1bb1115590ac45fdbcdc6d3f52aa6921/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:56.217649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6dbe1-74bd-4aea-8417-4298b009ece9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.218317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6dbe1-74bd-4aea-8417-4298b009ece9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.218630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6dbe1-74bd-4aea-8417-4298b009ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.246212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5bb6dbe1-74bd-4aea-8417-4298b009ece9 None None] MFA Rules not processed for user `617e3cd4a7f84e629d724160a6dcde61`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.247417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5bb6dbe1-74bd-4aea-8417-4298b009ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.270331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d339ea4-6461-479a-bc5a-c9e6bdaad0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.270988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339ea4-6461-479a-bc5a-c9e6bdaad0d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.271381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339ea4-6461-479a-bc5a-c9e6bdaad0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.271660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339ea4-6461-479a-bc5a-c9e6bdaad0d4 admin admin] PATH_INFO: `/v3/projects/e4960f1c54854f049c06fa301d9c117a/users/3752b1b002134a2c8de8b50999147bde/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.286428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d339ea4-6461-479a-bc5a-c9e6bdaad0d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4960f1c54854f049c06fa301d9c117a, user_id=3752b1b002134a2c8de8b50999147bde, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.289129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d339ea4-6461-479a-bc5a-c9e6bdaad0d4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.303656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 810/1610] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:56 2022] PUT /identity/v3/projects/e4960f1c54854f049c06fa301d9c117a/users/3752b1b002134a2c8de8b50999147bde/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:56.305053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aac6bc-3110-47ec-8939-c9f8586f839d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.305343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aac6bc-3110-47ec-8939-c9f8586f839d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.305781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aac6bc-3110-47ec-8939-c9f8586f839d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.317776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 801/1611] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.318933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6dbe1-74bd-4aea-8417-4298b009ece9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.328160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-43aac6bc-3110-47ec-8939-c9f8586f839d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.329399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-43aac6bc-3110-47ec-8939-c9f8586f839d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.337966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:56.383354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6dbe1-74bd-4aea-8417-4298b009ece9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.394470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:56.409776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 811/1612] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.410925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b85a-eca2-498d-b942-95ab7b2d39a2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.411215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b85a-eca2-498d-b942-95ab7b2d39a2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.411489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b85a-eca2-498d-b942-95ab7b2d39a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.430901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9d05b85a-eca2-498d-b942-95ab7b2d39a2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.432326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9d05b85a-eca2-498d-b942-95ab7b2d39a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.440144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cfc50-cf49-42b6-86a3-19e173421186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.440709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 812/1613] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.440846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cfc50-cf49-42b6-86a3-19e173421186 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.441040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cfc50-cf49-42b6-86a3-19e173421186 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.441315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cfc50-cf49-42b6-86a3-19e173421186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.441911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dfc20-5eca-4056-84b9-46fa3cd12e89 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.442196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dfc20-5eca-4056-84b9-46fa3cd12e89 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.442465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dfc20-5eca-4056-84b9-46fa3cd12e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.462705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-357dfc20-5eca-4056-84b9-46fa3cd12e89 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.464005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-357dfc20-5eca-4056-84b9-46fa3cd12e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.471868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 813/1614] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.472625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dfc20-5eca-4056-84b9-46fa3cd12e89 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.482042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dfc20-5eca-4056-84b9-46fa3cd12e89 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.487645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2199f55-e8c8-456e-8be5-94cb1a2218eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.488322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199f55-e8c8-456e-8be5-94cb1a2218eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.488646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199f55-e8c8-456e-8be5-94cb1a2218eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.488873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199f55-e8c8-456e-8be5-94cb1a2218eb admin admin] PATH_INFO: `/v3/projects/4ebb258de32f40c68f86e4b401efde5d/users/f1815cf2b3f347eb8276769cc30cf116/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.503922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199f55-e8c8-456e-8be5-94cb1a2218eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ebb258de32f40c68f86e4b401efde5d, user_id=f1815cf2b3f347eb8276769cc30cf116, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.506580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199f55-e8c8-456e-8be5-94cb1a2218eb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.507708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cfc50-cf49-42b6-86a3-19e173421186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.509164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cfc50-cf49-42b6-86a3-19e173421186 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.519993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 802/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:56.520940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 814/1616] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:03:56 2022] PUT /identity/v3/projects/4ebb258de32f40c68f86e4b401efde5d/users/f1815cf2b3f347eb8276769cc30cf116/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:03:56.525059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948bc2a-4f0d-457d-9530-632be503e638 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.525059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948bc2a-4f0d-457d-9530-632be503e638 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.525059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948bc2a-4f0d-457d-9530-632be503e638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.525314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.525314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.525314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.543743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a948bc2a-4f0d-457d-9530-632be503e638 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.544977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a948bc2a-4f0d-457d-9530-632be503e638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.555280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] MFA Rules not processed for user `3752b1b002134a2c8de8b50999147bde`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.556797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.612536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 803/1617] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.614149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c390-0c9c-4603-a2ca-6c333ee24723 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.614433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c390-0c9c-4603-a2ca-6c333ee24723 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.614706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c390-0c9c-4603-a2ca-6c333ee24723 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.622768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 804/1618] 10.222.0.52 () {54 vars in 961 bytes} [Thu Oct 6 11:03:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:03:56.623473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c390-0c9c-4603-a2ca-6c333ee24723 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.626896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-15f3c390-0c9c-4603-a2ca-6c333ee24723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.640555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 815/1619] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.641236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.642574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.689440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c390-0c9c-4603-a2ca-6c333ee24723 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.693282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.693879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.694165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.694438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.702877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ba196-9f03-4dc5-b240-3979e8d24181 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.703479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.705286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.706423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871dff5a-49e0-48a7-9e0c-6cefc41c7626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.707016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dff5a-49e0-48a7-9e0c-6cefc41c7626 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.707293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dff5a-49e0-48a7-9e0c-6cefc41c7626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.707559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dff5a-49e0-48a7-9e0c-6cefc41c7626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.714701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 805/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:56.715486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.716592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dff5a-49e0-48a7-9e0c-6cefc41c7626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.718646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dff5a-49e0-48a7-9e0c-6cefc41c7626 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.724026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d177ba-13de-4a72-ad27-4e10bd22efa5 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.727793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.728411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.728530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 816/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:56.728968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.729248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.729835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a87cf-04c0-477a-abcc-5b5b169cccb3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.730126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a87cf-04c0-477a-abcc-5b5b169cccb3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.730399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a87cf-04c0-477a-abcc-5b5b169cccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.736693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:56.744859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.746847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.750656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ca1a87cf-04c0-477a-abcc-5b5b169cccb3 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.751866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ca1a87cf-04c0-477a-abcc-5b5b169cccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.758614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 806/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:56.762209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.765170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 817/1623] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.766592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88047bbc-687b-4b7f-98f4-42bab234b40e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.766888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88047bbc-687b-4b7f-98f4-42bab234b40e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.767166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88047bbc-687b-4b7f-98f4-42bab234b40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.776048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1871b6-0110-4f9f-b118-e2c72f073eb4 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.776524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:56.781739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.782424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.782705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.783007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.793232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.794662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.806460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 807/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:56.807395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.818899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-88047bbc-687b-4b7f-98f4-42bab234b40e None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:56.820157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-88047bbc-687b-4b7f-98f4-42bab234b40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:56.887928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be2ca-3d4e-4920-90c7-348f13ec4637 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.897228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238b7c6-7dce-46de-8931-6db5a1718d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.898040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238b7c6-7dce-46de-8931-6db5a1718d17 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.898310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238b7c6-7dce-46de-8931-6db5a1718d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.900318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238b7c6-7dce-46de-8931-6db5a1718d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.924704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 818/1625] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Oct 6 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:56.925873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88047bbc-687b-4b7f-98f4-42bab234b40e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.938443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88047bbc-687b-4b7f-98f4-42bab234b40e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.943981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.944174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.944685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.945517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:56.959022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:56.961150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:56.975743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 819/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:56.976863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:56.988725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67643f78-7df3-4fcf-9f86-332845557630 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:56.994677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:56.996006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:56.996577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:56.996694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:57.011486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:57.013449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:57.018230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238b7c6-7dce-46de-8931-6db5a1718d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:57.022974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238b7c6-7dce-46de-8931-6db5a1718d17 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:57.032304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 820/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:57.033616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:57.048931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d28ac-f2ce-4cc4-852b-6a8ba4faa601 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:57.056537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65a420-8026-4b0f-a070-6fd5bc17a2f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:57.057542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65a420-8026-4b0f-a070-6fd5bc17a2f3 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:57.057947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65a420-8026-4b0f-a070-6fd5bc17a2f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:57.058391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65a420-8026-4b0f-a070-6fd5bc17a2f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:57.073632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 808/1628] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:57.173317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65a420-8026-4b0f-a070-6fd5bc17a2f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:57.178781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65a420-8026-4b0f-a070-6fd5bc17a2f3 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:57.206840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 821/1629] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:57.704546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721435-ba76-479f-a165-3e0336530a3b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:57.705166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721435-ba76-479f-a165-3e0336530a3b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:57.705166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721435-ba76-479f-a165-3e0336530a3b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:57.706561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 809/1630] 10.222.0.52 () {54 vars in 912 bytes} [Thu Oct 6 11:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:03:57.712406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ad527-b796-4ee2-b578-fb47b93295ed None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:57.712959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ad527-b796-4ee2-b578-fb47b93295ed None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:57.712959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ad527-b796-4ee2-b578-fb47b93295ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:57.737805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dc8ad527-b796-4ee2-b578-fb47b93295ed None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:57.739668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dc8ad527-b796-4ee2-b578-fb47b93295ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:57.757231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 822/1631] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:03:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:57.767563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90721435-ba76-479f-a165-3e0336530a3b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:57.786281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90721435-ba76-479f-a165-3e0336530a3b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:57.794848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c448e3-7bd9-4aa5-b697-d246a6ecfa86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:57.797133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c448e3-7bd9-4aa5-b697-d246a6ecfa86 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:57.798119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c448e3-7bd9-4aa5-b697-d246a6ecfa86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:57.798685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c448e3-7bd9-4aa5-b697-d246a6ecfa86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:57.831744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c448e3-7bd9-4aa5-b697-d246a6ecfa86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:57.833122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c448e3-7bd9-4aa5-b697-d246a6ecfa86 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:57.868525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 810/1632] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:03:59.098703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e65da-7d58-41ff-a235-a04574d0ec4c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.099258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e65da-7d58-41ff-a235-a04574d0ec4c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.099258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e65da-7d58-41ff-a235-a04574d0ec4c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.100922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 823/1633] 10.222.0.52 () {54 vars in 961 bytes} [Thu Oct 6 11:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:03:59.111967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125e85a-35d7-4208-aff3-d21a45462975 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.113216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125e85a-35d7-4208-aff3-d21a45462975 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.113882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125e85a-35d7-4208-aff3-d21a45462975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.147256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2125e85a-35d7-4208-aff3-d21a45462975 None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:59.150043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2125e85a-35d7-4208-aff3-d21a45462975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:59.169647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 811/1634] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Oct 6 11:03:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:59.170973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e062-b807-44cf-bfe3-8491660bb78d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.171241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e062-b807-44cf-bfe3-8491660bb78d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.172149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e062-b807-44cf-bfe3-8491660bb78d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.173697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 824/1635] 10.222.0.52 () {54 vars in 912 bytes} [Thu Oct 6 11:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 11:03:59.181282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68542da4-f3fd-44a9-af4e-5423be7c029d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.181573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68542da4-f3fd-44a9-af4e-5423be7c029d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.182019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68542da4-f3fd-44a9-af4e-5423be7c029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.220589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-68542da4-f3fd-44a9-af4e-5423be7c029d None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:59.222031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68542da4-f3fd-44a9-af4e-5423be7c029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:59.240252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 812/1636] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:03:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:59.247650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070e062-b807-44cf-bfe3-8491660bb78d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:59.271784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070e062-b807-44cf-bfe3-8491660bb78d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:59.277863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:59.278680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.280531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.281912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.294462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:59.296652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:59.320901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 825/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:03:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:03:59.666352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23681d1-4def-464d-aa4b-2aa637a91588 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.666706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23681d1-4def-464d-aa4b-2aa637a91588 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.666922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23681d1-4def-464d-aa4b-2aa637a91588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.701427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d23681d1-4def-464d-aa4b-2aa637a91588 None None] MFA Rules not processed for user `636eb68bb1314f4ba70b0e9ce039cc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:03:59.703632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d23681d1-4def-464d-aa4b-2aa637a91588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:03:59.774581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:03:59.784763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 813/1638] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:03:59 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:03:59.789611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:59.804653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbe13f-c9db-4ad3-aaeb-fe44b910f70d service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:59.812231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:59.813072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.813451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.813909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.814644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:59.816851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:59.856505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 826/1639] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:03:59 2022] POST /identity/v3/projects => generated 405 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:59.862029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23681d1-4def-464d-aa4b-2aa637a91588 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:03:59.877290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23681d1-4def-464d-aa4b-2aa637a91588 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:03:59.959767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:03:59.960438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:03:59.960882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:03:59.961203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:03:59.962116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:03:59.964656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:03:59.998193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 814/1640] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:03:59 2022] POST /identity/v3/users => generated 309 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:03:59.999827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:00.016079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b5157-1f8c-46a5-b309-e0476f5ae3fb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:00.024690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:00.025838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:00.031421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:00.031790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:00.032640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:00.037145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:00.047269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:00.056288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 827/1641] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:00.062207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:00.080180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5580ca-2617-42f6-ad96-af17adbe3dd9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:00.095339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:00.096184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:00.097464 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:00.097808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] PATH_INFO: `/v3/projects/3883e67ba21e4ee5b242577c1aa2b299/users/73a4ec0f8c2b40409a066b800a7cc9a1/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:00.120558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3883e67ba21e4ee5b242577c1aa2b299, user_id=73a4ec0f8c2b40409a066b800a7cc9a1, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:00.124591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:00.148408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 815/1642] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:00 2022] PUT /identity/v3/projects/3883e67ba21e4ee5b242577c1aa2b299/users/73a4ec0f8c2b40409a066b800a7cc9a1/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:00.152097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:00.173440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000514b7-9d6b-4115-835d-2a2646e6f2cb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:00.345781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:00.347713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:00.348032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:00.349196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:00.351958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:00.361985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:00.397133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 828/1643] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:00.398230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:00.414447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248cd32-3ba3-4a86-902f-91bddfc37726 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:00.448248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796fce-99f5-4d2e-b019-0bcd202e6995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:00.452543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796fce-99f5-4d2e-b019-0bcd202e6995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:00.452832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796fce-99f5-4d2e-b019-0bcd202e6995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:00.453112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796fce-99f5-4d2e-b019-0bcd202e6995 admin admin] PATH_INFO: `/v3/projects/3883e67ba21e4ee5b242577c1aa2b299/users/73a4ec0f8c2b40409a066b800a7cc9a1/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:00.466852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796fce-99f5-4d2e-b019-0bcd202e6995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3883e67ba21e4ee5b242577c1aa2b299, user_id=73a4ec0f8c2b40409a066b800a7cc9a1, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:00.480783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796fce-99f5-4d2e-b019-0bcd202e6995 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:00.520916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 816/1644] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:00 2022] PUT /identity/v3/projects/3883e67ba21e4ee5b242577c1aa2b299/users/73a4ec0f8c2b40409a066b800a7cc9a1/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:03.956819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:03.974853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99044e17-2f8c-4efc-9db3-e5ce102a9ec3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:04.300177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:04.301320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:04.302975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:04.304063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] PATH_INFO: `/v3/users/617e3cd4a7f84e629d724160a6dcde61` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:04.332625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] RBAC: Authorizing `identity:delete_user(user_id=617e3cd4a7f84e629d724160a6dcde61)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:04.334020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:04:04.334020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Oct 06 11:04:04.334020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: Oct 06 11:04:04.336737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:04.581526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617e3cd4a7f84e629d724160a6dcde61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:04.654846 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617e3cd4a7f84e629d724160a6dcde61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:04.655724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617e3cd4a7f84e629d724160a6dcde61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:04.671707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617e3cd4a7f84e629d724160a6dcde61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:04.671707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4d69056b-5e0f-4e17-8eb7-c30c856c076b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617e3cd4a7f84e629d724160a6dcde61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:04.714458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 829/1645] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:03 2022] DELETE /identity/v3/users/617e3cd4a7f84e629d724160a6dcde61 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:04.716254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:04.716760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:04.716995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:04.758005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:04.759474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:04.875940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761c8b-9b50-4c74-b2dd-9c6824d9cb29 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:04.876669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761c8b-9b50-4c74-b2dd-9c6824d9cb29 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:04.877236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761c8b-9b50-4c74-b2dd-9c6824d9cb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:04.887792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 817/1646] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:04.907605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:04.909501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:04.960412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bf761c8b-9b50-4c74-b2dd-9c6824d9cb29 None None] MFA Rules not processed for user `73a4ec0f8c2b40409a066b800a7cc9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:04.962551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bf761c8b-9b50-4c74-b2dd-9c6824d9cb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:04.972679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:05.060581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a865a-a92c-45d0-9234-894a11b9307a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:05.064805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:05.065570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:05.069375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:05.069894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:05.085492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:05.093941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:05.095592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:05.127618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 818/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:05.140488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 830/1648] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:04 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:07.300305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:07.302499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:07.324584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e39e1d-54c9-46d4-a50f-d235174371bf service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:07.337850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:07.338504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:07.339945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:07.340380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] PATH_INFO: `/v3/projects/69b37e6e45a4446093bd3bd7b3524a67` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:07.341778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b37e6e45a4446093bd3bd7b3524a67)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:07.344028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 11:04:07.344028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Oct 06 11:04:07.344028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: Oct 06 11:04:07.345710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:07.516462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b37e6e45a4446093bd3bd7b3524a67', 'request_id': 'req-b1a1d100-26d9-43bb-87f0-6c002823b13a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:07.537348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b37e6e45a4446093bd3bd7b3524a67', 'request_id': 'req-b1a1d100-26d9-43bb-87f0-6c002823b13a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:07.573526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b37e6e45a4446093bd3bd7b3524a67', 'request_id': 'req-b1a1d100-26d9-43bb-87f0-6c002823b13a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:07.605537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] The token cache is being invalidate because project 69b37e6e45a4446093bd3bd7b3524a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:07.605817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:07.607140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1a1d100-26d9-43bb-87f0-6c002823b13a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:07.609833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 819/1649] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:07 2022] DELETE /identity/v3/projects/69b37e6e45a4446093bd3bd7b3524a67 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:07.615408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7a11-cd0c-4359-a66b-fdef0e35137f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:07.615890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7a11-cd0c-4359-a66b-fdef0e35137f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:07.616187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7a11-cd0c-4359-a66b-fdef0e35137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:07.648941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-186b7a11-cd0c-4359-a66b-fdef0e35137f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:07.650661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-186b7a11-cd0c-4359-a66b-fdef0e35137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:07.839464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 831/1650] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:07.846363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d50b-c31f-4cae-8e72-7b36b75cbe9b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:07.847366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d50b-c31f-4cae-8e72-7b36b75cbe9b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:07.847773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d50b-c31f-4cae-8e72-7b36b75cbe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:07.915717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-51c2d50b-c31f-4cae-8e72-7b36b75cbe9b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:07.923309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-51c2d50b-c31f-4cae-8e72-7b36b75cbe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:07.961619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 820/1651] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:07.966256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b7a11-cd0c-4359-a66b-fdef0e35137f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:07.981119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b7a11-cd0c-4359-a66b-fdef0e35137f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:07.987384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:07.988702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:07.989192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:07.990552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:07.991564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:07.992142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:07.994901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:08.012350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 832/1652] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:08.016873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d50b-c31f-4cae-8e72-7b36b75cbe9b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:08.029447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d50b-c31f-4cae-8e72-7b36b75cbe9b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:08.037023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f43397-bdae-4b38-9adf-4e0f4cf94387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:08.037984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43397-bdae-4b38-9adf-4e0f4cf94387 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.038470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43397-bdae-4b38-9adf-4e0f4cf94387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.038954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43397-bdae-4b38-9adf-4e0f4cf94387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.040322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f43397-bdae-4b38-9adf-4e0f4cf94387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:08.042832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f43397-bdae-4b38-9adf-4e0f4cf94387 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:08.105815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 821/1653] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:08 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:08.112135 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:08.133584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ab76-72e6-48bc-8174-bf3d35308094 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:08.237106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e3e29-6398-4138-be2e-d4e0fe73743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:08.237999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e3e29-6398-4138-be2e-d4e0fe73743b admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.238431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e3e29-6398-4138-be2e-d4e0fe73743b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.238850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e3e29-6398-4138-be2e-d4e0fe73743b admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.239662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e3e29-6398-4138-be2e-d4e0fe73743b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:08.242069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e3e29-6398-4138-be2e-d4e0fe73743b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:08.276116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09a7f5-1278-4426-b18f-5c4ccc9bee46 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.276636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09a7f5-1278-4426-b18f-5c4ccc9bee46 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.276993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09a7f5-1278-4426-b18f-5c4ccc9bee46 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.281592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 822/1654] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:04:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:04:08.301703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 833/1655] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:08 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:08.305857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09a7f5-1278-4426-b18f-5c4ccc9bee46 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:08.338434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09a7f5-1278-4426-b18f-5c4ccc9bee46 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:08.342055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262cccd-642b-4925-a08e-9c75f68a55b9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.342455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262cccd-642b-4925-a08e-9c75f68a55b9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.342906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262cccd-642b-4925-a08e-9c75f68a55b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.367235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:08.370729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.373066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.373495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.375210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:08.382749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:08.407089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3262cccd-642b-4925-a08e-9c75f68a55b9 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:08.409562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3262cccd-642b-4925-a08e-9c75f68a55b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:08.430897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 823/1656] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:08.434572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:08.469519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:08.470257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a32eb5-2f18-4c93-9d65-2de52f2415cd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:08.488430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:08.491398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.492673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.493724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.498950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:08.507936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:08.554571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 824/1657] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:08.557029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:08.581304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e42fa8-0dde-426a-ae2a-2e558bdad1b3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:08.601429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:08.604446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.606345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.606767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] PATH_INFO: `/v3/projects/b008d145e73f4d0981ca4db3d6355e16/users/43b838abbe554d409b932a7b672dd185/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.661806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 834/1658] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 06 11:04:08.680280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b008d145e73f4d0981ca4db3d6355e16, user_id=43b838abbe554d409b932a7b672dd185, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:08.683851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:08.732698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 825/1659] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:08 2022] PUT /identity/v3/projects/b008d145e73f4d0981ca4db3d6355e16/users/43b838abbe554d409b932a7b672dd185/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:08.736873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:08.737344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:08.738057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:08.823167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] MFA Rules not processed for user `43b838abbe554d409b932a7b672dd185`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:08.825239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:08.981177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 835/1660] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:08.981301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:09.122164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8aaa32-d5ee-46b9-a04c-41c7111b49ca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:09.128571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:09.129377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.129807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.130228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.131078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:09.133295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:09.144367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:09.144623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:09.146164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:09.195179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 826/1661] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:08 2022] POST /identity/v3/projects => generated 381 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:09.200062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:09.308385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1a92-4159-413f-8f68-1bcd0f0f4f47 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:09.325161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:09.326072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.326495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.326926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.327943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:09.330944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0683f-7d29-4bf0-b5fe-390b85d7d24d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:09.331234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:09.373444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 827/1662] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:09 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:09.378606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:09.399749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d2e7-ebfa-41c4-aa8a-af5f3d60d3b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:09.407085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:09.408288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.409384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.409812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.410622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:09.413209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:09.427787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 828/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:09.432269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:09.441815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:09.448096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ff414-6b2b-4641-85e3-8993c99aa32f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:09.455229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:09.456346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.457024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.457445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] PATH_INFO: `/v3/projects/ef3b0d17cfb64b498f843e33fb8123a2/users/39cb83235ea74344a6ed96b404129c3d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.479565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:09.480100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3b0d17cfb64b498f843e33fb8123a2, user_id=39cb83235ea74344a6ed96b404129c3d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:09.480388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.480820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.481095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.484788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:09.506842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:09.509777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:09.517418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 829/1664] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:09 2022] PUT /identity/v3/projects/ef3b0d17cfb64b498f843e33fb8123a2/users/39cb83235ea74344a6ed96b404129c3d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:09.519486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:09.540536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 836/1665] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:09.676625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0bdd-4c2c-4664-9757-60bb7cae74c4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:09.692033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c946f-df77-465b-8f9f-f62186e41ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:09.692747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c946f-df77-465b-8f9f-f62186e41ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.693031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c946f-df77-465b-8f9f-f62186e41ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.693306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c946f-df77-465b-8f9f-f62186e41ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.694014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c946f-df77-465b-8f9f-f62186e41ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:09.695666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c946f-df77-465b-8f9f-f62186e41ab9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:09.722952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 830/1666] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:09.728022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:09.741745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02ee4d-b95d-4eec-8390-5703aa24a4a6 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:09.751301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:09.752159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.752553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.752832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] PATH_INFO: `/v3/projects/ef3b0d17cfb64b498f843e33fb8123a2/users/39cb83235ea74344a6ed96b404129c3d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.755372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3b0d17cfb64b498f843e33fb8123a2, user_id=39cb83235ea74344a6ed96b404129c3d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:09.762049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:09.785696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 837/1667] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:09 2022] PUT /identity/v3/projects/ef3b0d17cfb64b498f843e33fb8123a2/users/39cb83235ea74344a6ed96b404129c3d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:09.788010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ff0f2-4c8b-47bc-9e53-1bbfd941becb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:09.788310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ff0f2-4c8b-47bc-9e53-1bbfd941becb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:09.788595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ff0f2-4c8b-47bc-9e53-1bbfd941becb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:09.844103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c00ff0f2-4c8b-47bc-9e53-1bbfd941becb None None] MFA Rules not processed for user `39cb83235ea74344a6ed96b404129c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:09.846175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c00ff0f2-4c8b-47bc-9e53-1bbfd941becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:09.929369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:10.043861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 831/1668] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:10.185470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:10.197181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:10.446717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740447a-5d62-40cb-a4bd-e95a33d8901f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:10.462873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:10.463694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:10.464102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:10.464472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:10.485098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:10.486726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:10.510303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 838/1669] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:10.769621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ff0f2-4c8b-47bc-9e53-1bbfd941becb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:10.789840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ff0f2-4c8b-47bc-9e53-1bbfd941becb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:10.823653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93713808-a808-4893-b72d-1236e1c4104c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:10.824423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713808-a808-4893-b72d-1236e1c4104c service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:10.829130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713808-a808-4893-b72d-1236e1c4104c service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:10.829928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713808-a808-4893-b72d-1236e1c4104c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:11.039979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93713808-a808-4893-b72d-1236e1c4104c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:11.041608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93713808-a808-4893-b72d-1236e1c4104c service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:11.067728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 832/1670] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:11.334304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:11.335998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:11.461918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:11.462553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:11.463005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:11.532207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] MFA Rules not processed for user `1bb1115590ac45fdbcdc6d3f52aa6921`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:11.533681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:11.547555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab2271-b5eb-49e9-91ad-3d60cd382518 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:11.562357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:11.563721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:11.565646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:11.565949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] PATH_INFO: `/v3/users/3752b1b002134a2c8de8b50999147bde` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:11.568891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3752b1b002134a2c8de8b50999147bde)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:11.576466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:11.748703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 833/1671] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:11.757460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:11.761375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:11.782050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3752b1b002134a2c8de8b50999147bde'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:11.841015 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3752b1b002134a2c8de8b50999147bde'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:11.844722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3752b1b002134a2c8de8b50999147bde'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:11.858009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:11.880727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3752b1b002134a2c8de8b50999147bde'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:11.881223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ae6438d-164d-4d78-b150-b48137d5cc62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3752b1b002134a2c8de8b50999147bde'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:11.930863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 839/1672] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:11 2022] DELETE /identity/v3/users/3752b1b002134a2c8de8b50999147bde => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:11.939650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:11.940005 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:11.940288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:11.948522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:12.025356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:12.026612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:12.081267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7e07c-7ea1-4c9c-a54c-e648c1ed1294 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:12.265205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 840/1673] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:12.348423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:12.349354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:12.349779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:12.354737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:12.361043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:12.374116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:12.375682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:12.515646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:12.523149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:12.551030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 834/1674] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:12.561115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1054-1e0c-479a-9e45-eac850d6e84f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:12.575352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeab2b7-1b0a-4ad2-bb39-7a08db05a63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:12.576323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeab2b7-1b0a-4ad2-bb39-7a08db05a63a service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:12.576831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeab2b7-1b0a-4ad2-bb39-7a08db05a63a service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:12.577999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeab2b7-1b0a-4ad2-bb39-7a08db05a63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:12.591318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeab2b7-1b0a-4ad2-bb39-7a08db05a63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:12.594200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeab2b7-1b0a-4ad2-bb39-7a08db05a63a service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:12.615250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 841/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:12.713935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:12.715568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:12.741026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d08a3f-a3af-48b3-ac0c-b4e639fbc70d service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:12.758660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:12.758660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:12.758660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:12.758660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] PATH_INFO: `/v3/users/636eb68bb1314f4ba70b0e9ce039cc8d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:12.761270 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=636eb68bb1314f4ba70b0e9ce039cc8d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:12.763234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:12.912941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '636eb68bb1314f4ba70b0e9ce039cc8d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:12.952759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '636eb68bb1314f4ba70b0e9ce039cc8d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:12.970719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '636eb68bb1314f4ba70b0e9ce039cc8d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:12.971443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '636eb68bb1314f4ba70b0e9ce039cc8d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:12.982774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '636eb68bb1314f4ba70b0e9ce039cc8d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:12.983987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:13.007550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 835/1676] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:12 2022] DELETE /identity/v3/users/636eb68bb1314f4ba70b0e9ce039cc8d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:13.077191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:13.077486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:13.077885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:13.167981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] MFA Rules not processed for user `56835f0c8dc0416d9fa70662e40c856d`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:13.169819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:13.281452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 842/1677] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:13.296585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:13.299580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:13.498577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ef744-309a-4fd0-8588-ccc37412fb84 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:13.505845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f446aa9-36fe-4362-a4c9-6bb4e1ad1f46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:13.506559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f446aa9-36fe-4362-a4c9-6bb4e1ad1f46 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:13.506841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f446aa9-36fe-4362-a4c9-6bb4e1ad1f46 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:13.507110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f446aa9-36fe-4362-a4c9-6bb4e1ad1f46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:13.526147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f446aa9-36fe-4362-a4c9-6bb4e1ad1f46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:13.527973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f446aa9-36fe-4362-a4c9-6bb4e1ad1f46 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:13.541671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 836/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:13.643689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:13.645341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:13.702617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:13.860435 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d70c1-6ec8-4970-a4ea-5a36cee091d5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:13.873167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:13.877279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:13.878568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:13.879881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] PATH_INFO: `/v3/projects/353c183951e4445c9e31217081964962` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:13.882934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=353c183951e4445c9e31217081964962)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:13.885737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:14.031197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1d1bd-18b9-4fdb-b59d-f34af270cd5c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:14.032151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1d1bd-18b9-4fdb-b59d-f34af270cd5c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:14.032431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1d1bd-18b9-4fdb-b59d-f34af270cd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:14.094133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353c183951e4445c9e31217081964962', 'request_id': 'req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.111990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353c183951e4445c9e31217081964962', 'request_id': 'req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.115870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-62e1d1bd-18b9-4fdb-b59d-f34af270cd5c None None] MFA Rules not processed for user `f1815cf2b3f347eb8276769cc30cf116`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:14.122942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62e1d1bd-18b9-4fdb-b59d-f34af270cd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:14.128175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353c183951e4445c9e31217081964962', 'request_id': 'req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.175118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] The token cache is being invalidate because project 353c183951e4445c9e31217081964962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:14.175654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.177459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.183170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 843/1679] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:13 2022] DELETE /identity/v3/projects/353c183951e4445c9e31217081964962 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:14.187196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:14.189991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:14.367598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 837/1680] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:14.380719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:14.381291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:14.382813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:14.429483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefc51-ab87-43e9-af7f-94fa7dbbba24 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:14.439992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] MFA Rules not processed for user `f1815cf2b3f347eb8276769cc30cf116`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:14.440881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:14.441535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:14.441831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:14.442068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] PATH_INFO: `/v3/users/73a4ec0f8c2b40409a066b800a7cc9a1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:14.444406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:14.446708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a4ec0f8c2b40409a066b800a7cc9a1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:14.451629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:14.467338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 838/1681] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:14.477495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:14.480538 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:14.648637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a4ec0f8c2b40409a066b800a7cc9a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.651328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:14.680464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a4ec0f8c2b40409a066b800a7cc9a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.681436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a4ec0f8c2b40409a066b800a7cc9a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.704299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a4ec0f8c2b40409a066b800a7cc9a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.705395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a4ec0f8c2b40409a066b800a7cc9a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:14.758873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 844/1682] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:14 2022] DELETE /identity/v3/users/73a4ec0f8c2b40409a066b800a7cc9a1 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:14.758873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:14.758873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:14.775240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:14.795170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9869b2-cb43-41b2-bad4-c7c9e1508864 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:14.946471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f33b8-2fe7-4f5c-9617-f5f04faf0e14 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:14.957947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:14.958574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:14.958829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:14.959103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] PATH_INFO: `/v3/projects/e4960f1c54854f049c06fa301d9c117a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:14.961812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4960f1c54854f049c06fa301d9c117a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:14.964341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:14.974719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112654c-a6ad-4738-8342-39d2dbe689ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:14.975344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112654c-a6ad-4738-8342-39d2dbe689ef service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:14.975621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112654c-a6ad-4738-8342-39d2dbe689ef service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:14.975923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112654c-a6ad-4738-8342-39d2dbe689ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:14.977765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b112654c-a6ad-4738-8342-39d2dbe689ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:15.121485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4960f1c54854f049c06fa301d9c117a', 'request_id': 'req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.147144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4960f1c54854f049c06fa301d9c117a', 'request_id': 'req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.182854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4960f1c54854f049c06fa301d9c117a', 'request_id': 'req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.198798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112654c-a6ad-4738-8342-39d2dbe689ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:15.206890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112654c-a6ad-4738-8342-39d2dbe689ef service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:15.220967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] The token cache is being invalidate because project e4960f1c54854f049c06fa301d9c117a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:15.220967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.221979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.229285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 845/1683] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:14 2022] DELETE /identity/v3/projects/e4960f1c54854f049c06fa301d9c117a => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:15.235208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:15.244863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:15.256843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 839/1684] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 780 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:15.260559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e080af-556e-42cb-988d-3f40a5c49d17 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:15.265054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e080af-556e-42cb-988d-3f40a5c49d17 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:15.269305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e080af-556e-42cb-988d-3f40a5c49d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:15.341080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a9e080af-556e-42cb-988d-3f40a5c49d17 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:15.345245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a9e080af-556e-42cb-988d-3f40a5c49d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:15.480494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:15.486679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b99c0-5bb7-4e95-a77a-68beef7c959e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:15.505957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:15.506611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:15.507141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:15.507437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] PATH_INFO: `/v3/users/43b838abbe554d409b932a7b672dd185` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:15.519241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b838abbe554d409b932a7b672dd185)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:15.520923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:15.555933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 840/1685] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:15.564472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd712-0ac5-42ee-8703-cda259ed0849 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:15.567571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd712-0ac5-42ee-8703-cda259ed0849 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:15.567571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd712-0ac5-42ee-8703-cda259ed0849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:15.644889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-632cd712-0ac5-42ee-8703-cda259ed0849 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:15.646136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-632cd712-0ac5-42ee-8703-cda259ed0849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:15.653641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b838abbe554d409b932a7b672dd185'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.687937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b838abbe554d409b932a7b672dd185'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.691808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 841/1686] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:15.691808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:15.692558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b838abbe554d409b932a7b672dd185'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.692819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:15.695509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:15.714006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b838abbe554d409b932a7b672dd185'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.714809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b838abbe554d409b932a7b672dd185'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:15.752240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 846/1687] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:15 2022] DELETE /identity/v3/users/43b838abbe554d409b932a7b672dd185 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:15.752240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:15.756398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:15.822668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] MFA Rules not processed for user `02930aebe31e450f9fc989af369bf319`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:15.824040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:15.940565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:15.942968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1a0d3-767e-449a-ab98-a2ad19f0091a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:15.961660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a132b-286d-4ae8-9864-74623d879e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:15.962307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a132b-286d-4ae8-9864-74623d879e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:15.962588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a132b-286d-4ae8-9864-74623d879e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:15.962869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a132b-286d-4ae8-9864-74623d879e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:15.963716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a132b-286d-4ae8-9864-74623d879e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:15.964100 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a132b-286d-4ae8-9864-74623d879e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:15.965682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a132b-286d-4ae8-9864-74623d879e79 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:15.996549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 847/1688] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:15.997859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524933cc-2765-42f3-b3b9-ce08430cb056 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:15.998152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524933cc-2765-42f3-b3b9-ce08430cb056 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:15.998927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524933cc-2765-42f3-b3b9-ce08430cb056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.065317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-524933cc-2765-42f3-b3b9-ce08430cb056 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:16.067756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-524933cc-2765-42f3-b3b9-ce08430cb056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.079550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 848/1689] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:16.080871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524933cc-2765-42f3-b3b9-ce08430cb056 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:16.096702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524933cc-2765-42f3-b3b9-ce08430cb056 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:16.102126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 842/1690] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:16.112173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:16.112371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.118897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a3df9-93e7-4780-931e-e47ed7b73a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:16.120048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3df9-93e7-4780-931e-e47ed7b73a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.120617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3df9-93e7-4780-931e-e47ed7b73a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.121176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3df9-93e7-4780-931e-e47ed7b73a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.122079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a3df9-93e7-4780-931e-e47ed7b73a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:16.126172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a3df9-93e7-4780-931e-e47ed7b73a75 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:16.156286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa86b8-0a92-4de2-89fc-7145569e1e41 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:16.165180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:16.166250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.166746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.167360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] PATH_INFO: `/v3/projects/3883e67ba21e4ee5b242577c1aa2b299` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.181386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3883e67ba21e4ee5b242577c1aa2b299)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:16.183585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 849/1691] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:16 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:16.186881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.188086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:16.189247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.190966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.213552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] MFA Rules not processed for user `02930aebe31e450f9fc989af369bf319`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:16.215146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.318516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:16.332730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3883e67ba21e4ee5b242577c1aa2b299', 'request_id': 'req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:16.348540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3883e67ba21e4ee5b242577c1aa2b299', 'request_id': 'req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:16.358162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 850/1692] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:16.359095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:16.360184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3883e67ba21e4ee5b242577c1aa2b299', 'request_id': 'req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:16.366625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.392354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] The token cache is being invalidate because project 3883e67ba21e4ee5b242577c1aa2b299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:16.392354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:16.393467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:16.400696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 843/1693] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:16 2022] DELETE /identity/v3/projects/3883e67ba21e4ee5b242577c1aa2b299 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:16.401481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:16.404544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.502487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171bb67-c2b8-46af-99c9-63a9bd27f250 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:16.561367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ff1ac-fbdb-484e-8c2b-34d86b33a58d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:16.584636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:16.585330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.585683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.585966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.586676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:16.591692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:16.631774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3404c-caab-46df-909b-8b8f199671f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:16.632422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3404c-caab-46df-909b-8b8f199671f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.632703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3404c-caab-46df-909b-8b8f199671f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.632976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3404c-caab-46df-909b-8b8f199671f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.639995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f5d3404c-caab-46df-909b-8b8f199671f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.662974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3404c-caab-46df-909b-8b8f199671f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:16.664595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3404c-caab-46df-909b-8b8f199671f0 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:16.669317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 844/1694] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:16 2022] POST /identity/v3/users => generated 291 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:16.670496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:16.674559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.689900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 851/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:16.696578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20426896-4f35-47e9-af37-4d207a4f039e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.696578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20426896-4f35-47e9-af37-4d207a4f039e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.696861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20426896-4f35-47e9-af37-4d207a4f039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.743591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-20426896-4f35-47e9-af37-4d207a4f039e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:16.744864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-20426896-4f35-47e9-af37-4d207a4f039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.778574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 852/1696] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:16.778574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20426896-4f35-47e9-af37-4d207a4f039e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:16.794575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20426896-4f35-47e9-af37-4d207a4f039e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:16.798272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:16.807206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37246de6-73b7-46e4-afa1-3bfc88a01d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:16.808728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37246de6-73b7-46e4-afa1-3bfc88a01d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.810568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37246de6-73b7-46e4-afa1-3bfc88a01d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.810861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37246de6-73b7-46e4-afa1-3bfc88a01d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.815438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37246de6-73b7-46e4-afa1-3bfc88a01d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:16.819076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37246de6-73b7-46e4-afa1-3bfc88a01d4b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:16.851098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 853/1697] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:16.858610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f5db-b90b-4d56-9d15-503fb6d799b2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.859398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f5db-b90b-4d56-9d15-503fb6d799b2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.859398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f5db-b90b-4d56-9d15-503fb6d799b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.876918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333fd3-ece4-4f1b-8fdf-80ede6bf1a06 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:16.886651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:16.887336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.887594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.892169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.896092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.912622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-edf4f5db-b90b-4d56-9d15-503fb6d799b2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:16.915545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-edf4f5db-b90b-4d56-9d15-503fb6d799b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:16.941505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 854/1698] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:16.941505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f5db-b90b-4d56-9d15-503fb6d799b2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:16.965255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f5db-b90b-4d56-9d15-503fb6d799b2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:16.980463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:16.982597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:16.983825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:16.983825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:16.984520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:16.991970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.020434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 855/1699] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:17.022199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.050647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d5876-177b-44a2-abf6-57409912934e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:17.065068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.067213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.068916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb62bbc-c7c1-441c-b8b6-7b4ff5b9daf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:17.072636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb62bbc-c7c1-441c-b8b6-7b4ff5b9daf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.073151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb62bbc-c7c1-441c-b8b6-7b4ff5b9daf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.073679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb62bbc-c7c1-441c-b8b6-7b4ff5b9daf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.076664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb62bbc-c7c1-441c-b8b6-7b4ff5b9daf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:17.078802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb62bbc-c7c1-441c-b8b6-7b4ff5b9daf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.084274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb62bbc-c7c1-441c-b8b6-7b4ff5b9daf1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.089096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 845/1700] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:17.109156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.127725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 856/1701] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:17.133125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d526f-739c-4807-9c15-6d998d296a6d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.133749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d526f-739c-4807-9c15-6d998d296a6d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.134214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d526f-739c-4807-9c15-6d998d296a6d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.136096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 857/1702] 10.222.0.52 () {54 vars in 912 bytes} [Thu Oct 6 11:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:04:17.137032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d526f-739c-4807-9c15-6d998d296a6d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.156498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96ae77-85c3-41ec-9806-5dd72028f1fd service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:17.169475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d526f-739c-4807-9c15-6d998d296a6d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:17.171327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cca94-bb8f-4651-b93b-3d914b9c9124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:17.172298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cca94-bb8f-4651-b93b-3d914b9c9124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.172568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cca94-bb8f-4651-b93b-3d914b9c9124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.173000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cca94-bb8f-4651-b93b-3d914b9c9124 admin admin] PATH_INFO: `/v3/projects/d602948289d346b39d260edcfffd3171/users/38e1b220af024d7a9dd95c6a9c244bb6/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.181351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402ac45-f5b0-4d9f-b141-757cf65d8e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:17.182343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402ac45-f5b0-4d9f-b141-757cf65d8e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.187538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402ac45-f5b0-4d9f-b141-757cf65d8e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.188216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402ac45-f5b0-4d9f-b141-757cf65d8e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.189138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402ac45-f5b0-4d9f-b141-757cf65d8e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.191439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402ac45-f5b0-4d9f-b141-757cf65d8e7d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.212552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cca94-bb8f-4651-b93b-3d914b9c9124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d602948289d346b39d260edcfffd3171, user_id=38e1b220af024d7a9dd95c6a9c244bb6, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.217233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cca94-bb8f-4651-b93b-3d914b9c9124 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.244278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 846/1703] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:17 2022] PUT /identity/v3/projects/d602948289d346b39d260edcfffd3171/users/38e1b220af024d7a9dd95c6a9c244bb6/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:17.247583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937423-48c6-4db9-884c-961ec6e3232e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.248567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937423-48c6-4db9-884c-961ec6e3232e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.250127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937423-48c6-4db9-884c-961ec6e3232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.273902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 858/1704] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:17 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:17.275517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c17b41-b26f-458d-8902-ed77bef829e1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.276081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c17b41-b26f-458d-8902-ed77bef829e1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.278664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c17b41-b26f-458d-8902-ed77bef829e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.353111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a6c17b41-b26f-458d-8902-ed77bef829e1 None None] MFA Rules not processed for user `38e1b220af024d7a9dd95c6a9c244bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:17.354412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a6c17b41-b26f-458d-8902-ed77bef829e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:17.401680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d6937423-48c6-4db9-884c-961ec6e3232e None None] MFA Rules not processed for user `b5c892640b3e409b9a99b4ca1d34a964`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:17.403565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d6937423-48c6-4db9-884c-961ec6e3232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:17.414009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:17.505632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 847/1705] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:04:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:17.507053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6937423-48c6-4db9-884c-961ec6e3232e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.518720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 859/1706] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:17.524881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c17b41-b26f-458d-8902-ed77bef829e1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.572358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:17.594587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6937423-48c6-4db9-884c-961ec6e3232e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:17.599891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:17.600292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.600570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.600840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.602119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.604341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.640922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 848/1707] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:17 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:17.641860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.650220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:17.680085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8bca2-dbc4-4e4b-a276-dc84a588a462 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:17.680544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c17b41-b26f-458d-8902-ed77bef829e1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:17.691200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:17.691868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.692140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.693290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.693822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:17.694519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.695545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.696089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.696816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.709458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.714773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.716302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.740077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 849/1708] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:17.742508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.773741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302efc-b4ac-455a-9775-79a6619951d9 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:17.785854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 860/1709] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:17 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:17.789600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.796646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:17.797276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:17.797729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:17.798013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:17.799335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:17.804226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:17.837114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 850/1710] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:17.838076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:17.867712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdae2d-2e73-4ef8-9591-1937d6372b12 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:18.030459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:18.031181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.031540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.031821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.032503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:18.034114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:18.064304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f512a-90af-4cd9-9bb5-f7d25e1579c4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:18.078269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:18.079106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.079509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.079763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.080493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:18.084841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:18.100269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 851/1711] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:18.105025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:18.144654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 861/1712] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:17 2022] POST /identity/v3/users => generated 297 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:18.149254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:18.157302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af41b-79a5-4b80-9349-643484949f8a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:18.169705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d76f9f-aa6a-44c3-98f0-d72c71339774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:18.171047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76f9f-aa6a-44c3-98f0-d72c71339774 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.173421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76f9f-aa6a-44c3-98f0-d72c71339774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.173782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76f9f-aa6a-44c3-98f0-d72c71339774 admin admin] PATH_INFO: `/v3/projects/676a074ed5a84cbda9dba7714d4c0b9e/users/c681c6eda6724c7da31105cb06c3a8a7/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.205438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6db10d-ba3b-4e01-91e6-c333651882c3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:18.236532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:18.238047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.239292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76f9f-aa6a-44c3-98f0-d72c71339774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=676a074ed5a84cbda9dba7714d4c0b9e, user_id=c681c6eda6724c7da31105cb06c3a8a7, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:18.239919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.240401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.242248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:18.251457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76f9f-aa6a-44c3-98f0-d72c71339774 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:18.255115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:18.292872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 862/1713] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:18.297980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:18.319543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db31def-4a33-40ef-87e2-1f61debb528d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:18.324065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 852/1714] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:18 2022] PUT /identity/v3/projects/676a074ed5a84cbda9dba7714d4c0b9e/users/c681c6eda6724c7da31105cb06c3a8a7/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:18.345728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ba8d-c05b-4a8c-8575-51b2b8ef6c4e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.345728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ba8d-c05b-4a8c-8575-51b2b8ef6c4e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.345728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ba8d-c05b-4a8c-8575-51b2b8ef6c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.407760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:18.477482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ef16ba8d-c05b-4a8c-8575-51b2b8ef6c4e None None] MFA Rules not processed for user `c681c6eda6724c7da31105cb06c3a8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:18.481724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ef16ba8d-c05b-4a8c-8575-51b2b8ef6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:18.533120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:18.534622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.534947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.535235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] PATH_INFO: `/v3/projects/3427a87b438a415eac73105af1ace8ba/users/4390c34bfb2646fa83bd166523d05b0e/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.571594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3427a87b438a415eac73105af1ace8ba, user_id=4390c34bfb2646fa83bd166523d05b0e, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:18.576408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:18.606821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 863/1715] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:18 2022] PUT /identity/v3/projects/3427a87b438a415eac73105af1ace8ba/users/4390c34bfb2646fa83bd166523d05b0e/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:18.610156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:18.613401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:18.712411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 853/1716] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:18.715963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8fbfd-5fa5-42bf-b042-e8e62cf1ad9b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.716252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8fbfd-5fa5-42bf-b042-e8e62cf1ad9b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.716529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8fbfd-5fa5-42bf-b042-e8e62cf1ad9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.739775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b3f0-abba-43e3-99fb-bacffc5720c8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:18.757696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:18.758502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:18.758788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:18.759061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] PATH_INFO: `/v3/users/baf7d8e8af204cbd830a79f54fb680ac` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:18.760528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf7d8e8af204cbd830a79f54fb680ac)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:18.762710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:18.774410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c8e8fbfd-5fa5-42bf-b042-e8e62cf1ad9b None None] MFA Rules not processed for user `1bb1115590ac45fdbcdc6d3f52aa6921`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:18.782391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c8e8fbfd-5fa5-42bf-b042-e8e62cf1ad9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:18.800854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:18.937509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf7d8e8af204cbd830a79f54fb680ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:18.977093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 854/1717] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:18.982900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8fbfd-5fa5-42bf-b042-e8e62cf1ad9b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:19.003424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf7d8e8af204cbd830a79f54fb680ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:19.007632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8fbfd-5fa5-42bf-b042-e8e62cf1ad9b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:19.011087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf7d8e8af204cbd830a79f54fb680ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:19.023929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:19.024550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:19.024834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:19.025109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:19.025773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:19.031189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:19.041551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf7d8e8af204cbd830a79f54fb680ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:19.042394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf7d8e8af204cbd830a79f54fb680ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:19.069907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 855/1718] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:19.071098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:19.082223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:19.091447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 864/1719] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:18 2022] DELETE /identity/v3/users/baf7d8e8af204cbd830a79f54fb680ac => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:19.093227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:19.115266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56a8dc-31af-4793-b455-6b9ee046865b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:19.116738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:19.251444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:19.269815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d377e-2978-433e-a4c7-e8e4bd572230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:19.270453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d377e-2978-433e-a4c7-e8e4bd572230 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:19.270735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d377e-2978-433e-a4c7-e8e4bd572230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:19.271020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d377e-2978-433e-a4c7-e8e4bd572230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:19.271662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d377e-2978-433e-a4c7-e8e4bd572230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:19.282260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d377e-2978-433e-a4c7-e8e4bd572230 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:19.344587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb0233-aba3-4cf9-8059-438acfb4f300 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:19.352346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 856/1720] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:19 2022] POST /identity/v3/projects => generated 373 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:19.365226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:19.369977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:19.372790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:19.402999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:19.404681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:19.529674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a6dd7-d60b-481b-a784-db00c27176a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:19.531436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a6dd7-d60b-481b-a784-db00c27176a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:19.531785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a6dd7-d60b-481b-a784-db00c27176a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:19.532073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a6dd7-d60b-481b-a784-db00c27176a0 admin admin] PATH_INFO: `/v3/projects/3427a87b438a415eac73105af1ace8ba/users/4390c34bfb2646fa83bd166523d05b0e/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:19.539227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a6dd7-d60b-481b-a784-db00c27176a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3427a87b438a415eac73105af1ace8ba, user_id=4390c34bfb2646fa83bd166523d05b0e, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:19.541983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a6dd7-d60b-481b-a784-db00c27176a0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:19.559079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 857/1721] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:19.560042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:19.561679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:19.580918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 865/1722] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:19 2022] PUT /identity/v3/projects/3427a87b438a415eac73105af1ace8ba/users/4390c34bfb2646fa83bd166523d05b0e/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:19.585141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d644b7-8570-4afb-b597-701c3df2951a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:19.585433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d644b7-8570-4afb-b597-701c3df2951a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:19.585717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d644b7-8570-4afb-b597-701c3df2951a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:19.648604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a6d644b7-8570-4afb-b597-701c3df2951a None None] MFA Rules not processed for user `f1815cf2b3f347eb8276769cc30cf116`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:19.659383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a6d644b7-8570-4afb-b597-701c3df2951a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:19.660819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:19.722803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60903a83-eb69-4f3a-817a-45b83bd6fe72 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:19.839900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 866/1723] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:19.839900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d644b7-8570-4afb-b597-701c3df2951a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:19.873081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:19.874237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:19.874520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:19.874797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:19.876652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:19.902214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:19.905910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:19.928016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 858/1724] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:19.929238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:19.931483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.085474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d644b7-8570-4afb-b597-701c3df2951a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:20.087099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ab543-ad39-4148-ba09-b591799b8aff service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:20.099448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e98022-39b7-40fb-a3a5-0cb02fdd89ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:20.100851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326411d-749c-4a28-b009-215349d24c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:20.101275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e98022-39b7-40fb-a3a5-0cb02fdd89ea service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.101665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e98022-39b7-40fb-a3a5-0cb02fdd89ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.102043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e98022-39b7-40fb-a3a5-0cb02fdd89ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.102139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326411d-749c-4a28-b009-215349d24c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.102369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326411d-749c-4a28-b009-215349d24c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.102647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326411d-749c-4a28-b009-215349d24c70 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.103298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326411d-749c-4a28-b009-215349d24c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:20.106975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326411d-749c-4a28-b009-215349d24c70 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:20.128247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e98022-39b7-40fb-a3a5-0cb02fdd89ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:20.130046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e98022-39b7-40fb-a3a5-0cb02fdd89ea service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:20.149427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 859/1725] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:20.154398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61c40-aa41-4530-98fe-d9fb307c5480 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.154803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61c40-aa41-4530-98fe-d9fb307c5480 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.155784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61c40-aa41-4530-98fe-d9fb307c5480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.173961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 867/1726] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:19 2022] POST /identity/v3/users => generated 293 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:20.175505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe2eb-e365-4b94-9977-95b3598c7bee None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.181084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe2eb-e365-4b94-9977-95b3598c7bee None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.181429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe2eb-e365-4b94-9977-95b3598c7bee None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.185318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 868/1727] 10.222.0.52 () {54 vars in 912 bytes} [Thu Oct 6 11:04:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:04:20.189374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.189662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.189940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.228951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:20.241097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8ef61c40-aa41-4530-98fe-d9fb307c5480 None None] MFA Rules not processed for user `4390c34bfb2646fa83bd166523d05b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:20.241588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8ef61c40-aa41-4530-98fe-d9fb307c5480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.243292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] MFA Rules not processed for user `56835f0c8dc0416d9fa70662e40c856d`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:20.245987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.401533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 869/1728] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:20.401533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:20.407887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.408387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:20.441034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 860/1729] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:20.445927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61c40-aa41-4530-98fe-d9fb307c5480 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:20.448621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289e2cb-35dd-42ed-8b22-790c27e363b4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:20.455995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:20.457686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.458141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.458547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] PATH_INFO: `/v3/projects/b008d145e73f4d0981ca4db3d6355e16` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.460366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b008d145e73f4d0981ca4db3d6355e16)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:20.464192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:20.474033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61c40-aa41-4530-98fe-d9fb307c5480 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:20.479661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4000c2-b780-47a6-93d2-293501c55c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:20.480599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4000c2-b780-47a6-93d2-293501c55c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.480822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4000c2-b780-47a6-93d2-293501c55c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.481602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4000c2-b780-47a6-93d2-293501c55c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.506249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4000c2-b780-47a6-93d2-293501c55c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:20.508261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4000c2-b780-47a6-93d2-293501c55c5c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:20.538647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 861/1730] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:20.541510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.542313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.542746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.564115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b008d145e73f4d0981ca4db3d6355e16', 'request_id': 'req-47c99688-e4f4-4cce-a2e1-8e6895929b9c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:20.572554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b008d145e73f4d0981ca4db3d6355e16', 'request_id': 'req-47c99688-e4f4-4cce-a2e1-8e6895929b9c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:20.578757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] MFA Rules not processed for user `b5c892640b3e409b9a99b4ca1d34a964`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:20.580879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b008d145e73f4d0981ca4db3d6355e16', 'request_id': 'req-47c99688-e4f4-4cce-a2e1-8e6895929b9c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:20.581114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.601008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] The token cache is being invalidate because project b008d145e73f4d0981ca4db3d6355e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:20.601576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:20.603260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:20.606025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 870/1731] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:20 2022] DELETE /identity/v3/projects/b008d145e73f4d0981ca4db3d6355e16 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:20.607264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:20.609286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.696275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 862/1732] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:20.697565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:20.699697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.718473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c99688-e4f4-4cce-a2e1-8e6895929b9c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:20.729454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:20.730377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.730797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.732545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.737114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:20.887761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:20.945355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5a69-8c80-4273-98ea-9b7f4056d895 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:20.949132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:20.950608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:20.975536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:20.977419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:20.977708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:20.977999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] PATH_INFO: `/v3/projects/ab46a2e55f8f42688f9ae22c86f7bdc4/users/1133f0f0312c493ba06ddf64c2fad135/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:20.979755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 871/1733] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:20.980669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:20.988535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.039343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d38e9f-568e-45f5-9551-ea798235fd5b service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:21.041860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab46a2e55f8f42688f9ae22c86f7bdc4, user_id=1133f0f0312c493ba06ddf64c2fad135, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:21.044690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:21.060141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:21.060791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:21.061104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:21.061924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] PATH_INFO: `/v3/users/39cb83235ea74344a6ed96b404129c3d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:21.064088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39cb83235ea74344a6ed96b404129c3d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:21.065979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:21.088492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 863/1734] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:20 2022] PUT /identity/v3/projects/ab46a2e55f8f42688f9ae22c86f7bdc4/users/1133f0f0312c493ba06ddf64c2fad135/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:21.092002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:21.096093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.203383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39cb83235ea74344a6ed96b404129c3d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.255366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39cb83235ea74344a6ed96b404129c3d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.256160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39cb83235ea74344a6ed96b404129c3d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.278467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39cb83235ea74344a6ed96b404129c3d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.279712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39cb83235ea74344a6ed96b404129c3d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.297952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507b69d-4e77-4290-873e-dc913430d61a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:21.309915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:21.310552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:21.310833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:21.315046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:21.321543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.322893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 872/1735] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:20 2022] DELETE /identity/v3/users/39cb83235ea74344a6ed96b404129c3d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:21.323760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:21.332915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.354717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:21.356211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:21.401563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 864/1736] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:21.402806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:21.407186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.459218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:21.501615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62c396-b3d0-4f93-8520-2a16d7458c9b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:21.508997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:21.510066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:21.510352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:21.510628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] PATH_INFO: `/v3/projects/2c108c50400744fa8e39a9f770625031` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:21.526288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c108c50400744fa8e39a9f770625031)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:21.536271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:21.539683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011df5-aeb8-46b5-9b0e-31e2f341319d service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:21.547520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:21.548441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:21.548931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:21.549393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:21.550189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:21.552542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:21.572592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 865/1737] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:21.573872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:21.576651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.713223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de1bcb-1aad-4d99-8e31-4b45cbfd36b4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:21.724529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c108c50400744fa8e39a9f770625031', 'request_id': 'req-1852eb62-180f-43e2-8fda-63943605b173'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.751368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c108c50400744fa8e39a9f770625031', 'request_id': 'req-1852eb62-180f-43e2-8fda-63943605b173'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.755298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:21.770748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c108c50400744fa8e39a9f770625031', 'request_id': 'req-1852eb62-180f-43e2-8fda-63943605b173'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.808396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] The token cache is being invalidate because project 2c108c50400744fa8e39a9f770625031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:21.808792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.810353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:21.812528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 873/1738] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:21 2022] DELETE /identity/v3/projects/2c108c50400744fa8e39a9f770625031 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:21.816086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:21.818452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.833349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e738e70-ba87-40d9-8d2a-36d9fcd33075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:21.834571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e738e70-ba87-40d9-8d2a-36d9fcd33075 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:21.835011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e738e70-ba87-40d9-8d2a-36d9fcd33075 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:21.836493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e738e70-ba87-40d9-8d2a-36d9fcd33075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:21.844395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e738e70-ba87-40d9-8d2a-36d9fcd33075 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:21.940959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852eb62-180f-43e2-8fda-63943605b173 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:21.959699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:21.960422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:21.960700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:21.960975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] PATH_INFO: `/v3/projects/ab46a2e55f8f42688f9ae22c86f7bdc4/users/1133f0f0312c493ba06ddf64c2fad135/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:21.965144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab46a2e55f8f42688f9ae22c86f7bdc4, user_id=1133f0f0312c493ba06ddf64c2fad135, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:21.972125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:21.990926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e738e70-ba87-40d9-8d2a-36d9fcd33075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:21.992609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e738e70-ba87-40d9-8d2a-36d9fcd33075 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:21.998145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 874/1739] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:21 2022] PUT /identity/v3/projects/ab46a2e55f8f42688f9ae22c86f7bdc4/users/1133f0f0312c493ba06ddf64c2fad135/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:21.999745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:22.009456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:22.023507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 866/1740] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:22.025055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:22.025341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:22.026380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:22.091633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] MFA Rules not processed for user `1133f0f0312c493ba06ddf64c2fad135`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:22.093686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:22.124643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:22.204255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ae2b-3839-4eec-8ec0-29a439ba6713 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:22.225014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:22.226159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:22.226159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:22.226159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] PATH_INFO: `/v3/users/db5f1d51cc8e412ab01c6d693f496357` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:22.230290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5f1d51cc8e412ab01c6d693f496357)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:22.251528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:22.353328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 867/1741] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:22.366305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:22.367753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:22.431756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5f1d51cc8e412ab01c6d693f496357'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:22.476372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5f1d51cc8e412ab01c6d693f496357'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:22.477452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5f1d51cc8e412ab01c6d693f496357'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:22.505440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5f1d51cc8e412ab01c6d693f496357'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:22.506108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5f1d51cc8e412ab01c6d693f496357'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:22.569121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:22.603657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 875/1742] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:21 2022] DELETE /identity/v3/users/db5f1d51cc8e412ab01c6d693f496357 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:22.631183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fedc82-50a9-4331-9004-0b72d379a755 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:22.794421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:22.795120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:22.795308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:22.796424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:22.814078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:23.006150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:23.009884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:23.039895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 868/1743] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:23.853088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:23.854801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:24.071891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460eed7-775c-4fe1-aea8-6f577f574221 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:24.087112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:24.092271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:24.092613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:24.092925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] PATH_INFO: `/v3/projects/bcc4792bd74946afb4f48d18141578a9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:24.093483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:24.094989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:24.112380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc4792bd74946afb4f48d18141578a9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:24.125209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4a4d-e17d-4fe1-a7f4-7ad4663b79c4 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:24.127561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:24.149825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:24.150743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:24.151136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:24.151559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] PATH_INFO: `/v3/projects/ef3b0d17cfb64b498f843e33fb8123a2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:24.152872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3b0d17cfb64b498f843e33fb8123a2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:24.155513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:24.206961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:24.294896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc4792bd74946afb4f48d18141578a9', 'request_id': 'req-5cf69398-b389-4e05-aad9-cf4790a6d548'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.310359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc4792bd74946afb4f48d18141578a9', 'request_id': 'req-5cf69398-b389-4e05-aad9-cf4790a6d548'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.341560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc4792bd74946afb4f48d18141578a9', 'request_id': 'req-5cf69398-b389-4e05-aad9-cf4790a6d548'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.349558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3b0d17cfb64b498f843e33fb8123a2', 'request_id': 'req-d1f3b626-f161-46df-a607-843d4b601c0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.384749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] The token cache is being invalidate because project bcc4792bd74946afb4f48d18141578a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:24.384749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.391974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.394736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 876/1744] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:23 2022] DELETE /identity/v3/projects/bcc4792bd74946afb4f48d18141578a9 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:24.395831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3b0d17cfb64b498f843e33fb8123a2', 'request_id': 'req-d1f3b626-f161-46df-a607-843d4b601c0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.428954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:24.429113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3b0d17cfb64b498f843e33fb8123a2', 'request_id': 'req-d1f3b626-f161-46df-a607-843d4b601c0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.432673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:24.457587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] The token cache is being invalidate because project ef3b0d17cfb64b498f843e33fb8123a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:24.460073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.475012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d1f3b626-f161-46df-a607-843d4b601c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:24.480936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 869/1745] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:24 2022] DELETE /identity/v3/projects/ef3b0d17cfb64b498f843e33fb8123a2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:24.484627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3bfeb-1c6d-4b1a-876a-22baa4ae8c60 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:24.485166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3bfeb-1c6d-4b1a-876a-22baa4ae8c60 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:24.485593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3bfeb-1c6d-4b1a-876a-22baa4ae8c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:24.589818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-aff3bfeb-1c6d-4b1a-876a-22baa4ae8c60 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:24.591332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aff3bfeb-1c6d-4b1a-876a-22baa4ae8c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:24.650725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf69398-b389-4e05-aad9-cf4790a6d548 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:24.852609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 870/1746] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:24.852609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cad41b-d590-46b6-808d-c7e64a74cf9b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:24.852609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cad41b-d590-46b6-808d-c7e64a74cf9b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:24.852609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cad41b-d590-46b6-808d-c7e64a74cf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:24.895301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:24.897796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b6cad41b-d590-46b6-808d-c7e64a74cf9b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:24.899304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b6cad41b-d590-46b6-808d-c7e64a74cf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:24.944100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 871/1747] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:24.945220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf677c2-0c1e-471f-b84f-2d52dc2ba96b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:24.945501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf677c2-0c1e-471f-b84f-2d52dc2ba96b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:24.945776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf677c2-0c1e-471f-b84f-2d52dc2ba96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:24.988658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:24.989968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:24.997401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:24.997724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] PATH_INFO: `/v3/users/c681c6eda6724c7da31105cb06c3a8a7` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:25.000693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c681c6eda6724c7da31105cb06c3a8a7)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:25.006341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:25.093963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-adf677c2-0c1e-471f-b84f-2d52dc2ba96b None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:25.095742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-adf677c2-0c1e-471f-b84f-2d52dc2ba96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:25.149673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c681c6eda6724c7da31105cb06c3a8a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:25.175896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c681c6eda6724c7da31105cb06c3a8a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:25.177479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c681c6eda6724c7da31105cb06c3a8a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:25.192920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c681c6eda6724c7da31105cb06c3a8a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:25.193806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae67cd3d-3cfe-41e3-b161-517b62d0bfd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c681c6eda6724c7da31105cb06c3a8a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:25.246977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 877/1748] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:24 2022] DELETE /identity/v3/users/c681c6eda6724c7da31105cb06c3a8a7 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:25.252067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda985f-8736-421e-be32-d58f2b7772cd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:25.252492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda985f-8736-421e-be32-d58f2b7772cd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:25.252658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda985f-8736-421e-be32-d58f2b7772cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:25.354369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 872/1749] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:25.358669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3daac-82e4-4f14-8632-e9aa910b5293 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:25.359395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3daac-82e4-4f14-8632-e9aa910b5293 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:25.360158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3daac-82e4-4f14-8632-e9aa910b5293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:25.384033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9eda985f-8736-421e-be32-d58f2b7772cd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:25.391643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9eda985f-8736-421e-be32-d58f2b7772cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:25.393761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:25.429246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-97e3daac-82e4-4f14-8632-e9aa910b5293 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:25.430692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-97e3daac-82e4-4f14-8632-e9aa910b5293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:25.601859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 878/1750] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:25.601859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:25.601859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:25.601859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:25.648062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:25.650243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:25.654928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 873/1751] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:25.662256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3daac-82e4-4f14-8632-e9aa910b5293 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:25.689542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 879/1752] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:25.691416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3daac-82e4-4f14-8632-e9aa910b5293 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:25.693903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:25.696424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:25.705038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:25.705844 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:25.706243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:25.706764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:25.707554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:25.708001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:25.713179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:25.743970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 874/1753] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:25.745039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:25.766440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341aee04-37ea-48ac-8609-b39cf195c0af admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:25.784313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:25.785193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:25.785721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:25.786749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:25.787090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:25.787532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:25.790286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:25.868782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 875/1754] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:25.875395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:25.883018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:25.932954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:26.073622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2111c-bd07-4a1f-b605-49d44b1a3d56 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:26.105929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:26.106685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:26.107034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:26.107373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:26.216412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81544cd4-250f-4288-abb6-3cc73e4b332a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:26.235107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:26.238263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:26.240397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:26.242879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:26.275867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:26.277560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:26.279527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:26.284086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:26.317219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 880/1755] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:04:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:04:26.318167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:26.333495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 876/1756] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:26.334772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:26.342450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c8d2-723b-4d5a-af7c-6f0db362906f service swift] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:26.392958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:26.401373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:26.401373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:26.401373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:26.401857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f68f-7b95-40d5-b2d1-df4d4d9ac8f2 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:26.405817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:26.411928 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:26.427183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:26.427183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:26.428851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:26.430061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:26.431830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:26.446393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:26.556937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 881/1757] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:26 2022] POST /identity/v3/projects => generated 365 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:26.557981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:26.613697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 877/1758] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:26 2022] POST /identity/v3/projects => generated 377 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:26.617279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:26.827539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77fa79-6b71-44e8-b9c3-4ffe1ded3fc5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:26.921368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894acc62-4816-442b-989f-8914adf26a17 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:26.929541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:26.930313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:26.930700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:26.931079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:26.931985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:26.934530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:26.946249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:27.006099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 878/1759] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:26 2022] POST /identity/v3/users => generated 289 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:27.018663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:27.038724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:27.065395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:27.066010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.066293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.066604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.069152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:27.071202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34823a66-6aee-4d13-946c-61f50ece9345 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:27.072229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:27.105959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:27.110548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.110839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.111119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.111826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:27.120757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 882/1760] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:26 2022] POST /identity/v3/users => generated 283 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:27.126191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:27.127964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:27.145196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62d60d-d657-4ad1-85b4-7c495761052f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:27.167820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:27.171649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.172608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.175919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.178381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:27.185487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:27.215932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 879/1761] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:27.216979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:27.239742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 883/1762] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:27.247357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:27.263024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258f57-536f-4074-b649-cc42a0d39ffa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:27.279222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9f05b-5141-4371-a05c-f4ad05fc81f3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:27.287010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:27.287759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:27.288455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.288910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.289415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.290251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:27.292547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:27.296918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.297602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.298713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.301434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:27.307623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 884/1763] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:27.313631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:27.317348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:27.329540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f487-ebb4-487d-93bf-9093ebd6ddbd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:27.336506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9eb347-ace5-4af2-b1da-6ee4e3473261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:27.337811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eb347-ace5-4af2-b1da-6ee4e3473261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.338543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eb347-ace5-4af2-b1da-6ee4e3473261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.339085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eb347-ace5-4af2-b1da-6ee4e3473261 admin admin] PATH_INFO: `/v3/projects/128aacc4191a4d22a2138dc145acac1e/users/6c40c138c61e41f78779739ea74dc4c5/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.376656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 880/1764] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:27.377915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:27.383400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9eb347-ace5-4af2-b1da-6ee4e3473261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=128aacc4191a4d22a2138dc145acac1e, user_id=6c40c138c61e41f78779739ea74dc4c5, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:27.394188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9eb347-ace5-4af2-b1da-6ee4e3473261 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:27.420129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389853f-4418-4c80-8c15-954197b4be00 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:27.455345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 885/1765] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:27 2022] PUT /identity/v3/projects/128aacc4191a4d22a2138dc145acac1e/users/6c40c138c61e41f78779739ea74dc4c5/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:27.467493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fced1a-cdf8-4ad4-a3e1-1fbf4a2adbc7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.468263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fced1a-cdf8-4ad4-a3e1-1fbf4a2adbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.468691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fced1a-cdf8-4ad4-a3e1-1fbf4a2adbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.534584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-99fced1a-cdf8-4ad4-a3e1-1fbf4a2adbc7 None None] MFA Rules not processed for user `6c40c138c61e41f78779739ea74dc4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:27.535917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-99fced1a-cdf8-4ad4-a3e1-1fbf4a2adbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:27.596802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81b640-dfa7-4d53-aae6-59cba46e2082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:27.600395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81b640-dfa7-4d53-aae6-59cba46e2082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.600873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81b640-dfa7-4d53-aae6-59cba46e2082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.601302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81b640-dfa7-4d53-aae6-59cba46e2082 admin admin] PATH_INFO: `/v3/projects/9abf1c702dcb42d7a73d1df494d66055/users/cf5c18cfb5fc4fb791a5a446f225f50c/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.663650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81b640-dfa7-4d53-aae6-59cba46e2082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abf1c702dcb42d7a73d1df494d66055, user_id=cf5c18cfb5fc4fb791a5a446f225f50c, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:27.663650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81b640-dfa7-4d53-aae6-59cba46e2082 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:27.737467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 881/1766] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:27 2022] PUT /identity/v3/projects/9abf1c702dcb42d7a73d1df494d66055/users/cf5c18cfb5fc4fb791a5a446f225f50c/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:27.739179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94824c99-30a1-46e8-9d39-45dde8e908d9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:27.740371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94824c99-30a1-46e8-9d39-45dde8e908d9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:27.741893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94824c99-30a1-46e8-9d39-45dde8e908d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:27.814177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 886/1767] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:27.819749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fced1a-cdf8-4ad4-a3e1-1fbf4a2adbc7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:27.861243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fced1a-cdf8-4ad4-a3e1-1fbf4a2adbc7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:27.873129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-94824c99-30a1-46e8-9d39-45dde8e908d9 None None] MFA Rules not processed for user `cf5c18cfb5fc4fb791a5a446f225f50c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:27.877201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-94824c99-30a1-46e8-9d39-45dde8e908d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:27.998746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.002116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.003993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.004494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.008714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.013744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.037558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 882/1768] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:28.041870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94824c99-30a1-46e8-9d39-45dde8e908d9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.057415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94824c99-30a1-46e8-9d39-45dde8e908d9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.084250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.085017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.085189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.085730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.086440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.089344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 887/1769] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:27 2022] POST /identity/v3/projects => generated 365 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:28.089841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.091393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.143904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 883/1770] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:28 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:28.143904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.149184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:28.220664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:28.254749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8841ff7-8c8f-4e4a-bab7-92ed659d7f32 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.338276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c8115-19c8-4355-a723-87f5b8c3665f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.358723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.359669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.360152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.360576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.361380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.363478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.370129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.370890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.371271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.371639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.372533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.375652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.414306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 884/1771] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:28 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:28.417332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 888/1772] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:28 2022] POST /identity/v3/users => generated 289 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:28.417927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.418687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.448024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63ecd9-3e00-430d-a853-f001b20deb8f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.448235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b144ae-a3c8-46d1-97c0-1d82204fb111 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.448336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.448336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.448336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.448599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.449490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.453916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.453916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.453916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.454212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.454293 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.454366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.454440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.471003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 889/1773] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:28.479406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 885/1774] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:28.484348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.485302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.508120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1a88f-24ef-4788-9802-e3556bc755f3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.513108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.513825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.514166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.514480 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.515168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.516805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.522631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5800cd-bc1b-419c-bd5e-636ffe1bade1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.548315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.549379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.549891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.550231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] PATH_INFO: `/v3/projects/f4de68e8cf9f494aaa9222b8a335f0cc/users/7ad07c8d0a5d489ead604af553d2e733/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.553032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 886/1775] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:28.554014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.572299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a25d22-e9c4-4488-9210-c12812837da3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.584074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7ab1b-c48c-4368-a330-7de0a657fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.585044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7ab1b-c48c-4368-a330-7de0a657fc64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.585492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7ab1b-c48c-4368-a330-7de0a657fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.590910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7ab1b-c48c-4368-a330-7de0a657fc64 admin admin] PATH_INFO: `/v3/projects/41ac3197669841c0851535aebf6724f4/users/62c8471abe3d44cc801120ab46772a33/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.593790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4de68e8cf9f494aaa9222b8a335f0cc, user_id=7ad07c8d0a5d489ead604af553d2e733, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.597719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.628450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7ab1b-c48c-4368-a330-7de0a657fc64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ac3197669841c0851535aebf6724f4, user_id=62c8471abe3d44cc801120ab46772a33, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.633733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7ab1b-c48c-4368-a330-7de0a657fc64 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.637701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 890/1776] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:28 2022] PUT /identity/v3/projects/f4de68e8cf9f494aaa9222b8a335f0cc/users/7ad07c8d0a5d489ead604af553d2e733/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:28.641742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.667636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 887/1777] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:28 2022] PUT /identity/v3/projects/41ac3197669841c0851535aebf6724f4/users/62c8471abe3d44cc801120ab46772a33/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:28.679037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4536e8-037b-4dd9-b38f-9290e329d1e4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.679580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4536e8-037b-4dd9-b38f-9290e329d1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.680072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4536e8-037b-4dd9-b38f-9290e329d1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.800703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fa4536e8-037b-4dd9-b38f-9290e329d1e4 None None] MFA Rules not processed for user `62c8471abe3d44cc801120ab46772a33`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:28.807338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fa4536e8-037b-4dd9-b38f-9290e329d1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:28.811854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db721f8-2376-43aa-8701-caa54ee0e6b9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:28.850079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:28.959970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:28.960661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:28.961009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:28.961288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:28.962887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:28.965360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:28.987975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 891/1778] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:28.987975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:28.990180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.011192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 888/1779] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:29.014960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4536e8-037b-4dd9-b38f-9290e329d1e4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.020710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc78d-5adc-44a1-af37-598f362f513d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.026230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4536e8-037b-4dd9-b38f-9290e329d1e4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.031007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.032587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.032885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.033168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] PATH_INFO: `/v3/projects/676a074ed5a84cbda9dba7714d4c0b9e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.034520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=676a074ed5a84cbda9dba7714d4c0b9e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.035142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.035960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.037103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.037235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] PATH_INFO: `/v3/projects/f4de68e8cf9f494aaa9222b8a335f0cc/users/7ad07c8d0a5d489ead604af553d2e733/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.037721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.046597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4de68e8cf9f494aaa9222b8a335f0cc, user_id=7ad07c8d0a5d489ead604af553d2e733, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.049412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.068794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 889/1780] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:29 2022] PUT /identity/v3/projects/f4de68e8cf9f494aaa9222b8a335f0cc/users/7ad07c8d0a5d489ead604af553d2e733/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:29.070204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.125126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676a074ed5a84cbda9dba7714d4c0b9e', 'request_id': 'req-5795f26f-b740-4d0e-8650-ab00a387ca20'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.127945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:29.132209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676a074ed5a84cbda9dba7714d4c0b9e', 'request_id': 'req-5795f26f-b740-4d0e-8650-ab00a387ca20'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.140355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676a074ed5a84cbda9dba7714d4c0b9e', 'request_id': 'req-5795f26f-b740-4d0e-8650-ab00a387ca20'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.154592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] The token cache is being invalidate because project 676a074ed5a84cbda9dba7714d4c0b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:29.155890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.156607 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5795f26f-b740-4d0e-8650-ab00a387ca20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.158254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 892/1781] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:28 2022] DELETE /identity/v3/projects/676a074ed5a84cbda9dba7714d4c0b9e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:29.159477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.159811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.160121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.160514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10133deb-c8cf-4e75-91c2-327574f4252d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.192990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] MFA Rules not processed for user `7ad07c8d0a5d489ead604af553d2e733`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:29.195541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.275347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.276433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.276723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.276995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.277635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.294434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.324023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 893/1782] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:29.324023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.326507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.339997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 890/1783] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:29 2022] POST /identity/v3/projects => generated 377 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:29.341027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.345307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.348394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3f6c-dd0f-4df9-b278-1ac54079366c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.383878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:29.422441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6448f4-7e27-46ba-b775-c5ae1cab78a8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.426485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.427468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.427758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.428069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.441359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.442287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.443001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.443434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] PATH_INFO: `/v3/users/1133f0f0312c493ba06ddf64c2fad135` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.444912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1133f0f0312c493ba06ddf64c2fad135)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.448616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.530436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.532072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.548591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1133f0f0312c493ba06ddf64c2fad135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.558431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 891/1784] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:29.559136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.564830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.580020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1133f0f0312c493ba06ddf64c2fad135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.580889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1133f0f0312c493ba06ddf64c2fad135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.594027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ecef3-4b60-487c-b081-3a46d91a80c8 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.603869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1133f0f0312c493ba06ddf64c2fad135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.603869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1133f0f0312c493ba06ddf64c2fad135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:29.616009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.617500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.617978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.618319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.619374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.621518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.635920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 894/1785] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:29 2022] DELETE /identity/v3/users/1133f0f0312c493ba06ddf64c2fad135 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:29.663182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 892/1786] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:29 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:29.664213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.666305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.693294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.696096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.707030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:29.713901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae0cd7-0b96-4a3a-b451-65133433f276 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.761990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332702f-69ed-417b-8521-24657bb7796f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.768832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.769539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.769819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.770095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.770802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.772798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.784114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 895/1787] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:29.789040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.801526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.802185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.802466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.802878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.805012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:29.807394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c10be-f755-4b2d-bca1-c39edb1ac6ad admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:29.830298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:29.831160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:29.831725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:29.832098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] PATH_INFO: `/v3/projects/ed7f469999aa4742a4f724f65b19a321/users/b1baed55913b47a3ab723fa51c6cfdc0/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:29.855792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f469999aa4742a4f724f65b19a321, user_id=b1baed55913b47a3ab723fa51c6cfdc0, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.859084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.882954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 896/1788] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:29 2022] PUT /identity/v3/projects/ed7f469999aa4742a4f724f65b19a321/users/b1baed55913b47a3ab723fa51c6cfdc0/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:29.886812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:29.900300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:29.903111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:29.915990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 893/1789] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:29.919453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:29.994760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420ef5-5be9-473a-b1ed-3cde21c077f8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.012791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fadb6-c762-4748-8e20-1e6bed47a4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.013573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fadb6-c762-4748-8e20-1e6bed47a4ff admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.013955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fadb6-c762-4748-8e20-1e6bed47a4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.014422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fadb6-c762-4748-8e20-1e6bed47a4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.015764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fadb6-c762-4748-8e20-1e6bed47a4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.017591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fadb6-c762-4748-8e20-1e6bed47a4ff admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.041764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 897/1790] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:30.045191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.066378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b578c5-3668-468d-8902-6a72f51a7f74 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.079130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.080470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.081291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.081802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] PATH_INFO: `/v3/projects/ed7f469999aa4742a4f724f65b19a321/users/b1baed55913b47a3ab723fa51c6cfdc0/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.084921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f469999aa4742a4f724f65b19a321, user_id=b1baed55913b47a3ab723fa51c6cfdc0, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.096730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.126556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 894/1791] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:30 2022] PUT /identity/v3/projects/ed7f469999aa4742a4f724f65b19a321/users/b1baed55913b47a3ab723fa51c6cfdc0/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:30.134315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0f18-2289-48c7-b455-1c302887f2fa None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.134833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0f18-2289-48c7-b455-1c302887f2fa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.135061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0f18-2289-48c7-b455-1c302887f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.173171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-262d0f18-2289-48c7-b455-1c302887f2fa None None] MFA Rules not processed for user `b1baed55913b47a3ab723fa51c6cfdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:30.174397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-262d0f18-2289-48c7-b455-1c302887f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:30.224503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.226395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:30.257451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 898/1792] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:30.269090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d0f18-2289-48c7-b455-1c302887f2fa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.302545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abe263-00cf-4f6c-b32f-299aca596e2b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.308020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.308653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.308928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.309202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] PATH_INFO: `/v3/projects/ab46a2e55f8f42688f9ae22c86f7bdc4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.310387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab46a2e55f8f42688f9ae22c86f7bdc4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.312510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.327823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d0f18-2289-48c7-b455-1c302887f2fa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.331456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.332094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.332376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.332676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.342721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.344607 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.355096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 899/1793] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:30.363456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:30.366704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab46a2e55f8f42688f9ae22c86f7bdc4', 'request_id': 'req-50f92243-bd9e-4819-93a4-351c6d2b59bf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:30.376949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab46a2e55f8f42688f9ae22c86f7bdc4', 'request_id': 'req-50f92243-bd9e-4819-93a4-351c6d2b59bf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:30.384226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab46a2e55f8f42688f9ae22c86f7bdc4', 'request_id': 'req-50f92243-bd9e-4819-93a4-351c6d2b59bf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:30.401415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.401778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] The token cache is being invalidate because project ab46a2e55f8f42688f9ae22c86f7bdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:30.402060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:30.402750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-50f92243-bd9e-4819-93a4-351c6d2b59bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:30.405222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 895/1794] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:30 2022] DELETE /identity/v3/projects/ab46a2e55f8f42688f9ae22c86f7bdc4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:30.406088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:30.409870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3040c-d35b-42a1-a0ef-dd7ca1b6fd7f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.410309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3040c-d35b-42a1-a0ef-dd7ca1b6fd7f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.410715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3040c-d35b-42a1-a0ef-dd7ca1b6fd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.417277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617123de-82dc-44f5-a644-86a5bf7d1138 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.433726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ee3040c-d35b-42a1-a0ef-dd7ca1b6fd7f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:30.435079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ee3040c-d35b-42a1-a0ef-dd7ca1b6fd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:30.466547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:30.477604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.478491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.478815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.479162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.481477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:30.505504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 896/1795] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:30.513110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979a4d-8d6c-4318-8c26-6a388e677af8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.513517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979a4d-8d6c-4318-8c26-6a388e677af8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.513911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979a4d-8d6c-4318-8c26-6a388e677af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.548532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b4979a4d-8d6c-4318-8c26-6a388e677af8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:30.550122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b4979a4d-8d6c-4318-8c26-6a388e677af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:30.558959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 897/1796] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:30.564395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979a4d-8d6c-4318-8c26-6a388e677af8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.567244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.568727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.575308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979a4d-8d6c-4318-8c26-6a388e677af8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.579091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 900/1797] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:30.581225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.581850 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.582129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.582407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.583068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:30.583379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.585027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.604061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 898/1798] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:30.606784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.625226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23d3c4-4611-4b3f-9c30-037502e4da3f service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.631335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.632076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.632364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.632692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.639828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.639828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.686766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 901/1799] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:30 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:30.690809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.710668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992ddfa-ded5-49b9-9a29-e441ea4c0f75 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.784661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.785365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.785694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.785934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.786586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.788615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.811636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 899/1800] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:30 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:30.815830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.827377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de309c-53c3-4bcb-b195-b8d08a39594a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.832185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.832777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.833067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.833342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.833986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.835594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.846837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 902/1801] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:30.851055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.862201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96114f4f-0a74-42ba-bcc2-e31342d57480 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.867169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56559b4-b4b9-4816-b685-5b508a9024fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.867973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56559b4-b4b9-4816-b685-5b508a9024fb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.868406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56559b4-b4b9-4816-b685-5b508a9024fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.868939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56559b4-b4b9-4816-b685-5b508a9024fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.869834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56559b4-b4b9-4816-b685-5b508a9024fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.872008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56559b4-b4b9-4816-b685-5b508a9024fb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.883933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 900/1802] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:30.888157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:30.900153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda78670-489e-4ccd-a668-7bf483ed72d4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:30.905841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:30.906703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.906984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.907256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] PATH_INFO: `/v3/projects/6de799cf25d04709a5d88e3d755aad6a/users/c0734b9ac75648609f7a4ce17a48d9ce/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:30.943219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de799cf25d04709a5d88e3d755aad6a, user_id=c0734b9ac75648609f7a4ce17a48d9ce, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:30.946266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:30.968609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 903/1803] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:30 2022] PUT /identity/v3/projects/6de799cf25d04709a5d88e3d755aad6a/users/c0734b9ac75648609f7a4ce17a48d9ce/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:30.992111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6937-8540-4eb6-8db3-6dba58476637 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:30.992111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6937-8540-4eb6-8db3-6dba58476637 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:30.992111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6937-8540-4eb6-8db3-6dba58476637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.011095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e60d6937-8540-4eb6-8db3-6dba58476637 None None] MFA Rules not processed for user `c0734b9ac75648609f7a4ce17a48d9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:31.012361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e60d6937-8540-4eb6-8db3-6dba58476637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:31.034533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:31.086039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 901/1804] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:31.090887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:31.164426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e0b91-2aa0-48ef-9198-01f3c078a15c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:31.173048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:31.176019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.176349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.177102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.177884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:31.181243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:31.223931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 904/1805] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:31 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:31.226942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6937-8540-4eb6-8db3-6dba58476637 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:31.238933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6937-8540-4eb6-8db3-6dba58476637 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:31.303649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:31.304449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.304743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.305019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.305662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:31.307598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:31.333501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 902/1806] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:31 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:31.334932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:31.346038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e73ac8-48dd-4a06-97c8-d96e0179ba08 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:31.350672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:31.351291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.351568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.352020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.352530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:31.354131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:31.366080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 905/1807] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:31.370477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:31.380527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71d8e6-e0fc-4abf-b272-3f3d7cce9f53 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:31.391156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:31.391156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.391156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.391156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] PATH_INFO: `/v3/projects/501fdf19d46b4f92b9ae34d01fa2fddd/users/3fb1e7ba68574d95956dfa2ec3741b44/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.408482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=501fdf19d46b4f92b9ae34d01fa2fddd, user_id=3fb1e7ba68574d95956dfa2ec3741b44, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:31.411069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:31.425538 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 903/1808] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:31 2022] PUT /identity/v3/projects/501fdf19d46b4f92b9ae34d01fa2fddd/users/3fb1e7ba68574d95956dfa2ec3741b44/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:31.429572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:31.440830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42c74e-f858-4885-b654-b1938e2c0eac admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:31.453960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:31.502883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334ca1-b1f0-4f85-bed4-b4c09c21f0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:31.503497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334ca1-b1f0-4f85-bed4-b4c09c21f0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.503779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334ca1-b1f0-4f85-bed4-b4c09c21f0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.504088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334ca1-b1f0-4f85-bed4-b4c09c21f0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.504734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334ca1-b1f0-4f85-bed4-b4c09c21f0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:31.506338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334ca1-b1f0-4f85-bed4-b4c09c21f0d6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:31.517311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 906/1809] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:31.521637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:31.534968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26384ff-5ff7-48b0-a1f9-507c07b119b6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:31.539773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:31.540577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.540859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.541136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] PATH_INFO: `/v3/projects/501fdf19d46b4f92b9ae34d01fa2fddd/users/3fb1e7ba68574d95956dfa2ec3741b44/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.547613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=501fdf19d46b4f92b9ae34d01fa2fddd, user_id=3fb1e7ba68574d95956dfa2ec3741b44, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:31.550357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:31.562601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 904/1810] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:31 2022] PUT /identity/v3/projects/501fdf19d46b4f92b9ae34d01fa2fddd/users/3fb1e7ba68574d95956dfa2ec3741b44/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:31.574449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.574743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.575024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:31.629706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] MFA Rules not processed for user `3fb1e7ba68574d95956dfa2ec3741b44`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:31.630966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:31.754995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 907/1811] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:31.768876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:31.772020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:31.905603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9d4c4-ab5e-40ed-980f-44fc9b8e5a48 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:31.919657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75eb96-fd83-4072-88a7-e79786122c8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:31.921317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75eb96-fd83-4072-88a7-e79786122c8b service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:31.921610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75eb96-fd83-4072-88a7-e79786122c8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:31.921882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75eb96-fd83-4072-88a7-e79786122c8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:32.001036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:32.068177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75eb96-fd83-4072-88a7-e79786122c8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:32.069911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75eb96-fd83-4072-88a7-e79786122c8b service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:32.080770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 905/1812] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:41.743558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=56 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:42.076395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=14 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:44.406946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:44.408402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:44.473242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dc996-e190-40b7-9c00-9422d879e3be None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:44.477732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:44.478355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:44.478635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:44.478909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] PATH_INFO: `/v3/users/cf5c18cfb5fc4fb791a5a446f225f50c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:44.480048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5c18cfb5fc4fb791a5a446f225f50c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:44.481980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:44.527766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5c18cfb5fc4fb791a5a446f225f50c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:44.539639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5c18cfb5fc4fb791a5a446f225f50c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:44.540323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5c18cfb5fc4fb791a5a446f225f50c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:44.548724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5c18cfb5fc4fb791a5a446f225f50c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:44.549326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5c18cfb5fc4fb791a5a446f225f50c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:44.561784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 908/1813] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:44 2022] DELETE /identity/v3/users/cf5c18cfb5fc4fb791a5a446f225f50c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:44.566803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:44.567331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:44.567444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:44.586054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:44.587241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:44.644774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 906/1814] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:44.654628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:44.656031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:44.712628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786c881-bcee-41f8-8223-5beb6a882674 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:44.716183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:44.716773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:44.717050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:44.717321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:44.726033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:44.727447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:44.737258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 909/1815] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:45.061616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:45.065382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:45.079411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d28c32-4460-41f6-aac5-558772bf9725 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:45.084288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:45.085053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:45.085208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:45.085474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] PATH_INFO: `/v3/projects/9abf1c702dcb42d7a73d1df494d66055` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:45.086600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9abf1c702dcb42d7a73d1df494d66055)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:45.088588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:45.139560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9abf1c702dcb42d7a73d1df494d66055', 'request_id': 'req-7395e642-4880-4f18-9ca3-71622d67711e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.145348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9abf1c702dcb42d7a73d1df494d66055', 'request_id': 'req-7395e642-4880-4f18-9ca3-71622d67711e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.152557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9abf1c702dcb42d7a73d1df494d66055', 'request_id': 'req-7395e642-4880-4f18-9ca3-71622d67711e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.166317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] The token cache is being invalidate because project 9abf1c702dcb42d7a73d1df494d66055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:45.166603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.167181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.168861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 907/1816] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:45 2022] DELETE /identity/v3/projects/9abf1c702dcb42d7a73d1df494d66055 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:45.173660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:45.183323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79f9dd-8088-4dad-9bdc-a6aea738ca67 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:45.202296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:45.247994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:45.248775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:45.248974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:45.249708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] PATH_INFO: `/v3/users/62c8471abe3d44cc801120ab46772a33` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:45.251469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c8471abe3d44cc801120ab46772a33)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:45.254525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:45.311586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c8471abe3d44cc801120ab46772a33'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.323812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c8471abe3d44cc801120ab46772a33'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.324552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c8471abe3d44cc801120ab46772a33'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.337564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c8471abe3d44cc801120ab46772a33'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.338236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c8471abe3d44cc801120ab46772a33'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.351340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 910/1817] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:45 2022] DELETE /identity/v3/users/62c8471abe3d44cc801120ab46772a33 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:45.619000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:45.620616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:45.637225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395e642-4880-4f18-9ca3-71622d67711e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:45.693525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:45.694213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:45.694443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:45.694720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] PATH_INFO: `/v3/projects/41ac3197669841c0851535aebf6724f4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:45.695830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ac3197669841c0851535aebf6724f4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:45.697755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:45.727339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:45.748030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ac3197669841c0851535aebf6724f4', 'request_id': 'req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.753949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ac3197669841c0851535aebf6724f4', 'request_id': 'req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.760192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ac3197669841c0851535aebf6724f4', 'request_id': 'req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.773292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] The token cache is being invalidate because project 41ac3197669841c0851535aebf6724f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:45.773580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.774198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.775679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 908/1818] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:45 2022] DELETE /identity/v3/projects/41ac3197669841c0851535aebf6724f4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:45.779738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:45.781224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:45.793733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc08aa-5f94-4c59-a1a1-1718fb656a61 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:45.849205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:45.849837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:45.850119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:45.850394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] PATH_INFO: `/v3/users/b1baed55913b47a3ab723fa51c6cfdc0` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:45.851507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1baed55913b47a3ab723fa51c6cfdc0)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:45.853458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:45.898159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:45.901034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1baed55913b47a3ab723fa51c6cfdc0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.916916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1baed55913b47a3ab723fa51c6cfdc0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.917812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1baed55913b47a3ab723fa51c6cfdc0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.925868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1baed55913b47a3ab723fa51c6cfdc0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.926613 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bddf006d-a476-4992-bb82-d7ce1954369d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1baed55913b47a3ab723fa51c6cfdc0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:45.939655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 911/1819] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:45 2022] DELETE /identity/v3/users/b1baed55913b47a3ab723fa51c6cfdc0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:46.243274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.244792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:46.264844 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300dc1f-5060-4885-83bc-9a9e7a3cc4be admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:46.348914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:46.349655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.350161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.350700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] PATH_INFO: `/v3/projects/ed7f469999aa4742a4f724f65b19a321` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.351867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7f469999aa4742a4f724f65b19a321)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:46.353955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:46.420231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7f469999aa4742a4f724f65b19a321', 'request_id': 'req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:46.425866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7f469999aa4742a4f724f65b19a321', 'request_id': 'req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:46.432242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7f469999aa4742a4f724f65b19a321', 'request_id': 'req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:46.445592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] The token cache is being invalidate because project ed7f469999aa4742a4f724f65b19a321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:46.445885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:46.446518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-56cc67f8-df84-45ed-97d4-31eb45d3c0ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:46.448045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 909/1820] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:46 2022] DELETE /identity/v3/projects/ed7f469999aa4742a4f724f65b19a321 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:46.455012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429f327-4ac3-42a1-94f7-194c534a71fd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.455385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429f327-4ac3-42a1-94f7-194c534a71fd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.455729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429f327-4ac3-42a1-94f7-194c534a71fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.475322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5429f327-4ac3-42a1-94f7-194c534a71fd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:46.476537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5429f327-4ac3-42a1-94f7-194c534a71fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:46.534580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 912/1821] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:46.542857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e7255-12b1-4f4b-b1cb-f5e2e7030169 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.543152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e7255-12b1-4f4b-b1cb-f5e2e7030169 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.543437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e7255-12b1-4f4b-b1cb-f5e2e7030169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.563795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b14e7255-12b1-4f4b-b1cb-f5e2e7030169 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:46.565033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b14e7255-12b1-4f4b-b1cb-f5e2e7030169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:46.572932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 910/1822] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:46.577082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429f327-4ac3-42a1-94f7-194c534a71fd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.587267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429f327-4ac3-42a1-94f7-194c534a71fd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:46.591749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:46.592378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.592658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.592929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.593668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:46.594163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:46.595883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:46.607246 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 913/1823] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:04:46.611623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e7255-12b1-4f4b-b1cb-f5e2e7030169 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.620936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e7255-12b1-4f4b-b1cb-f5e2e7030169 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:46.626213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:46.626839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.627123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.627398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.628067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:46.630012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:46.659076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 911/1824] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:46 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:46.663249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.673673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d76953-33f0-4bc7-a6cf-c62824fb6230 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:46.728782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:46.729306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.729582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.729852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.730482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:46.732425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:46.754665 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 914/1825] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:46 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:46.758739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.769642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ce6f7-a300-439b-be00-abb35fa83e8f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:46.774404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:46.774998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.775274 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.775547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.776219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:46.777791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:46.788169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 912/1826] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:46.792244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.802148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeb3a4-6463-4604-980f-71eb6af1a6d3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:46.806784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b4f88-057e-4379-9295-6517411dcd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:46.807378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b4f88-057e-4379-9295-6517411dcd10 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.807661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b4f88-057e-4379-9295-6517411dcd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.807979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b4f88-057e-4379-9295-6517411dcd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.808651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b4f88-057e-4379-9295-6517411dcd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:46.810257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b4f88-057e-4379-9295-6517411dcd10 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:46.820626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 915/1827] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:46.824649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.831328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:46.835130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad26b43-dfd9-46d0-9d52-8b0dfea99e32 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:46.840710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:46.841514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.841951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.842427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] PATH_INFO: `/v3/projects/988b46704eed4dea920dbd0db5c1dd0b/users/f5d3b9aad894446ba60090dde495571c/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.857906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=988b46704eed4dea920dbd0db5c1dd0b, user_id=f5d3b9aad894446ba60090dde495571c, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:46.860922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:46.873601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 913/1828] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:46 2022] PUT /identity/v3/projects/988b46704eed4dea920dbd0db5c1dd0b/users/f5d3b9aad894446ba60090dde495571c/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:46.878809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcf8e5-bcbb-4d4c-86a6-5fa03d628b7f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:46.879126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcf8e5-bcbb-4d4c-86a6-5fa03d628b7f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:46.879409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcf8e5-bcbb-4d4c-86a6-5fa03d628b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:46.900331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-10dcf8e5-bcbb-4d4c-86a6-5fa03d628b7f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:46.901513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-10dcf8e5-bcbb-4d4c-86a6-5fa03d628b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:46.922377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:46.965451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 916/1829] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:46.975640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:46.977450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:47.037368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385643-6785-4b74-97fe-d094832e23e3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:47.040975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:47.041578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:47.041857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:47.042145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:47.051440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:47.052938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:47.062336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 914/1830] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:47.885930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334aaca-5ba8-4d2e-bfe3-99875af8402a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:47.886393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334aaca-5ba8-4d2e-bfe3-99875af8402a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:47.886521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334aaca-5ba8-4d2e-bfe3-99875af8402a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:47.916503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8334aaca-5ba8-4d2e-bfe3-99875af8402a None None] MFA Rules not processed for user `f5d3b9aad894446ba60090dde495571c`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:47.917978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8334aaca-5ba8-4d2e-bfe3-99875af8402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:47.995292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 917/1831] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:48.000773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.011862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e831aa6-fd88-41e5-95c6-45df2b2b6650 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.022026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.022962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.023496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.023903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.024649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.026694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.065185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 915/1832] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:47 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:48.069931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334aaca-5ba8-4d2e-bfe3-99875af8402a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.083676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334aaca-5ba8-4d2e-bfe3-99875af8402a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.160331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.160946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.161231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.161510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.162144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.164276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.188130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 918/1833] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:48 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:48.192173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.202551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d824b-0544-4471-adce-23dfadab8dec admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.207541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.208432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.208719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.209024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.209647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.211383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.224267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 916/1834] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:48.228533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.240245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a7c5c-b28a-4465-a92c-c3dbbd7e531f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.244990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.245654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.245935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.246209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] PATH_INFO: `/v3/projects/afda2a77aa3c48cc946e64ca4ffc1e83/users/f51678a19f8f4f7bb62990809729d8b4/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.261655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afda2a77aa3c48cc946e64ca4ffc1e83, user_id=f51678a19f8f4f7bb62990809729d8b4, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.264379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.280856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 919/1835] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:48 2022] PUT /identity/v3/projects/afda2a77aa3c48cc946e64ca4ffc1e83/users/f51678a19f8f4f7bb62990809729d8b4/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:48.286254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.301824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4ad7d-2ab8-4b77-a265-cc187218aa41 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.373931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0422a-86d3-4807-99dc-c9e9cf5c939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.374813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0422a-86d3-4807-99dc-c9e9cf5c939e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.375048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0422a-86d3-4807-99dc-c9e9cf5c939e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.375323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0422a-86d3-4807-99dc-c9e9cf5c939e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.376149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0422a-86d3-4807-99dc-c9e9cf5c939e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.377685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0422a-86d3-4807-99dc-c9e9cf5c939e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.391880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 917/1836] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:48.397108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.412299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a64ec-6043-4e46-b877-383b8016ea04 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.421054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887d826-eccb-478a-8a48-6c6c8e275676 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.421474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887d826-eccb-478a-8a48-6c6c8e275676 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.421923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887d826-eccb-478a-8a48-6c6c8e275676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.424052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.424952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.425618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.426148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] PATH_INFO: `/v3/projects/afda2a77aa3c48cc946e64ca4ffc1e83/users/f51678a19f8f4f7bb62990809729d8b4/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.429272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] RBAC: Authorizing `identity:create_grant(project_id=afda2a77aa3c48cc946e64ca4ffc1e83, user_id=f51678a19f8f4f7bb62990809729d8b4, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.434662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.446920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 920/1837] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:48 2022] PUT /identity/v3/projects/afda2a77aa3c48cc946e64ca4ffc1e83/users/f51678a19f8f4f7bb62990809729d8b4/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:48.460155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3887d826-eccb-478a-8a48-6c6c8e275676 None None] MFA Rules not processed for user `b5c892640b3e409b9a99b4ca1d34a964`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:48.462049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3887d826-eccb-478a-8a48-6c6c8e275676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:48.517780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:48.530632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 918/1838] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Oct 6 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:48.569630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.572190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:48.585024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887d826-eccb-478a-8a48-6c6c8e275676 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.597217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887d826-eccb-478a-8a48-6c6c8e275676 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.607173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.608011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.608292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.608564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.616622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cd35b-cf45-44be-8e27-f5eb19b2344c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.621673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.624102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.625929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.626574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.626855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.627130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.636264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:48.639642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.642471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.655372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 919/1839] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:48.657345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.667896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e218c5-9aa7-4dc6-bad6-4159e06240cb service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.674685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.675934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.676221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.676499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.680310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 921/1840] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:48.681624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:48.688005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.689496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.699334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 920/1841] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:48.708827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9accdce-3364-4373-9d32-2313c1eda829 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:48.713068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:48.713983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:48.714337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:48.714866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:48.723588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:48.726390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:48.755926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 922/1842] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:49.370164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:49.371708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:49.425131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:49.426589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:49.485083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10961-9b25-4fda-9c1d-a87c7860f389 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:49.491509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:49.492563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:49.492563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:49.492979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] PATH_INFO: `/v3/users/38e1b220af024d7a9dd95c6a9c244bb6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:49.494349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e1b220af024d7a9dd95c6a9c244bb6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:49.496581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:49.582474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e1b220af024d7a9dd95c6a9c244bb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:49.593352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1b88-eae3-4ea5-8438-95a397b03976 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:49.604039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e1b220af024d7a9dd95c6a9c244bb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:49.604715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:49.605737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:49.606500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:49.606939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:49.612561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:49.620953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e1b220af024d7a9dd95c6a9c244bb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:49.621965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e1b220af024d7a9dd95c6a9c244bb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:49.632577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ebfd224-7ab2-438c-865e-b2fdf9e8016a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e1b220af024d7a9dd95c6a9c244bb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:49.640053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 921/1843] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:49 2022] DELETE /identity/v3/users/38e1b220af024d7a9dd95c6a9c244bb6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:49.645175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01204e00-6829-4823-bb9a-3047ef71c279 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:49.645716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01204e00-6829-4823-bb9a-3047ef71c279 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:49.646308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01204e00-6829-4823-bb9a-3047ef71c279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:49.692703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-01204e00-6829-4823-bb9a-3047ef71c279 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:49.694792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-01204e00-6829-4823-bb9a-3047ef71c279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:49.737388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:49.740138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:49.764749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 923/1844] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:04:49.790321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:49.817961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 922/1845] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:49.850118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:49.862364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562436b9-b75e-46a4-a501-6fdb00f0305f service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:49.954876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9925d8f-6efb-4999-addd-0d3a9be3e7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:49.955497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9925d8f-6efb-4999-addd-0d3a9be3e7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:49.955774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9925d8f-6efb-4999-addd-0d3a9be3e7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:49.956164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9925d8f-6efb-4999-addd-0d3a9be3e7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:49.985835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9925d8f-6efb-4999-addd-0d3a9be3e7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:49.987732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9925d8f-6efb-4999-addd-0d3a9be3e7d8 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:50.017714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 924/1846] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:50.962715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1ee1-e632-436f-ae33-abd6f6aace47 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:50.964233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1ee1-e632-436f-ae33-abd6f6aace47 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:50.964529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1ee1-e632-436f-ae33-abd6f6aace47 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:50.966938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 923/1847] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:04:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:04:50.989913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5c397-27c4-4daa-a709-dfa3a8a751ba None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:50.990471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5c397-27c4-4daa-a709-dfa3a8a751ba None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:50.991138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5c397-27c4-4daa-a709-dfa3a8a751ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:51.040390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0cd5c397-27c4-4daa-a709-dfa3a8a751ba None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:51.041839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0cd5c397-27c4-4daa-a709-dfa3a8a751ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:51.087495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:51.142120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 925/1848] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:04:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:51.164119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba1ee1-e632-436f-ae33-abd6f6aace47 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:51.165994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a4ba1ee1-e632-436f-ae33-abd6f6aace47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:51.195997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba1ee1-e632-436f-ae33-abd6f6aace47 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:51.207186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fe561-9c96-4d5e-a137-fda0f800b78e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:51.207513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fe561-9c96-4d5e-a137-fda0f800b78e service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:51.207758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fe561-9c96-4d5e-a137-fda0f800b78e service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:51.208119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fe561-9c96-4d5e-a137-fda0f800b78e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:51.279821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fe561-9c96-4d5e-a137-fda0f800b78e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:51.283385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fe561-9c96-4d5e-a137-fda0f800b78e service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:51.339268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 924/1849] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:51.687687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:51.687687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:51.687687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:51.729045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] MFA Rules not processed for user `f51678a19f8f4f7bb62990809729d8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:51.730480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:51.754697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79eb2c-bd10-4082-863b-8976fe57fe66 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:51.755006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79eb2c-bd10-4082-863b-8976fe57fe66 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:51.755280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79eb2c-bd10-4082-863b-8976fe57fe66 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:51.756893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 925/1850] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:04:51.773292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79eb2c-bd10-4082-863b-8976fe57fe66 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:51.775203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2d79eb2c-bd10-4082-863b-8976fe57fe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:51.793530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79eb2c-bd10-4082-863b-8976fe57fe66 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:51.799370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:51.800425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:51.801064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:51.801560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] PATH_INFO: `/v3/projects/d602948289d346b39d260edcfffd3171` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:51.803493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d602948289d346b39d260edcfffd3171)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:51.805950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:51.847904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 926/1851] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:51.861413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:51.863155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:51.886893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd602948289d346b39d260edcfffd3171', 'request_id': 'req-859e97db-3b33-4753-b6c4-993a2a2fa04b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:51.896418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd602948289d346b39d260edcfffd3171', 'request_id': 'req-859e97db-3b33-4753-b6c4-993a2a2fa04b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:51.912510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd602948289d346b39d260edcfffd3171', 'request_id': 'req-859e97db-3b33-4753-b6c4-993a2a2fa04b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:51.933322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] The token cache is being invalidate because project d602948289d346b39d260edcfffd3171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:51.933621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:51.934308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:51.936202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 926/1852] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:51 2022] DELETE /identity/v3/projects/d602948289d346b39d260edcfffd3171 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:51.941254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:51.943129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:51.969007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512987a-be93-41f5-9991-ad84b49b12c1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:52.057402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e97db-3b33-4753-b6c4-993a2a2fa04b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:52.072736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:52.073438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:52.073795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:52.074069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] PATH_INFO: `/v3/users/4390c34bfb2646fa83bd166523d05b0e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:52.076601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4390c34bfb2646fa83bd166523d05b0e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:52.080057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:52.080235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:52.082830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:52.084003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:52.085689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:52.093277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:52.138408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:52.153432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4390c34bfb2646fa83bd166523d05b0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:52.170500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:52.176008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4390c34bfb2646fa83bd166523d05b0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:52.191606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4390c34bfb2646fa83bd166523d05b0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:52.192636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4390c34bfb2646fa83bd166523d05b0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:52.200823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:52.202843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:52.203201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98332c7c-c0cd-4fee-8fe0-b6d3a36eed0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4390c34bfb2646fa83bd166523d05b0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:52.212782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 927/1853] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:51 2022] DELETE /identity/v3/users/4390c34bfb2646fa83bd166523d05b0e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:52.217790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:52.218301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:52.219510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 927/1854] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:52.220572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:52.249042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:52.251207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:52.374734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 928/1855] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:52.404009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:52.405713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:52.536313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a1bd-148c-4d02-bbaa-341a9630cc70 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:52.544109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32095d03-78a9-4010-a724-b530918778b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:52.544912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32095d03-78a9-4010-a724-b530918778b4 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:52.545496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32095d03-78a9-4010-a724-b530918778b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:52.545897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32095d03-78a9-4010-a724-b530918778b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:52.559175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32095d03-78a9-4010-a724-b530918778b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:52.561741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32095d03-78a9-4010-a724-b530918778b4 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:52.581534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 928/1856] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:52.908040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:52.909561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:52.978731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c5d8-8ab2-4324-8ce5-f30073660c56 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:52.983246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:52.983900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:52.984721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:52.984998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] PATH_INFO: `/v3/projects/3427a87b438a415eac73105af1ace8ba` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:52.986116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3427a87b438a415eac73105af1ace8ba)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:52.988048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.035389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3427a87b438a415eac73105af1ace8ba', 'request_id': 'req-de27355e-1cef-4277-a478-6cd1ba1abb3a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:53.040885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3427a87b438a415eac73105af1ace8ba', 'request_id': 'req-de27355e-1cef-4277-a478-6cd1ba1abb3a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:53.047801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3427a87b438a415eac73105af1ace8ba', 'request_id': 'req-de27355e-1cef-4277-a478-6cd1ba1abb3a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:53.056936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:53.062042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] The token cache is being invalidate because project 3427a87b438a415eac73105af1ace8ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:53.062678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:53.063429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de27355e-1cef-4277-a478-6cd1ba1abb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:53.065639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 929/1857] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:52 2022] DELETE /identity/v3/projects/3427a87b438a415eac73105af1ace8ba => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:53.070698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407abd20-d618-4a94-be33-5db9365c8202 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.071157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407abd20-d618-4a94-be33-5db9365c8202 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.071414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407abd20-d618-4a94-be33-5db9365c8202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.092978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-407abd20-d618-4a94-be33-5db9365c8202 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:53.094876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-407abd20-d618-4a94-be33-5db9365c8202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:53.152954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 929/1858] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:53.162029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db672909-6bf4-4a5b-9432-5e6ce6a5f6e2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.162315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db672909-6bf4-4a5b-9432-5e6ce6a5f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.162592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db672909-6bf4-4a5b-9432-5e6ce6a5f6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.182150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-db672909-6bf4-4a5b-9432-5e6ce6a5f6e2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:53.183286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db672909-6bf4-4a5b-9432-5e6ce6a5f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:53.191600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 930/1859] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:53.195788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407abd20-d618-4a94-be33-5db9365c8202 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.206250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407abd20-d618-4a94-be33-5db9365c8202 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.210697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.211312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.211588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.211878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.212520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:53.212817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.214405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.226109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 930/1860] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:53.230366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db672909-6bf4-4a5b-9432-5e6ce6a5f6e2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.239523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db672909-6bf4-4a5b-9432-5e6ce6a5f6e2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.244077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.244674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.244953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.245220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.245855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.247785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.281028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 931/1861] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:53 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:53.282074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.292117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857dd31-1212-4447-a4ed-ee41375b3c94 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.326511 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:53.353916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.354663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.355074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.355416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.356210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.359343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.392755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 931/1862] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:53 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:53.396468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.407532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900de87e-765a-4e04-9109-6f186e029420 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.412766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.413488 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.413876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.414248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.415089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.416951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.427784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 932/1863] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:53.431660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.443396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d056c-c716-4781-b30e-6c97e3d5fc50 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.448987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dd2f0-cc46-46f4-abad-9630cbc9d430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.449734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dd2f0-cc46-46f4-abad-9630cbc9d430 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.450122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dd2f0-cc46-46f4-abad-9630cbc9d430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.450513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dd2f0-cc46-46f4-abad-9630cbc9d430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.451466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dd2f0-cc46-46f4-abad-9630cbc9d430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.453827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dd2f0-cc46-46f4-abad-9630cbc9d430 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.466656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 932/1864] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:53.471071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.494238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1a73a-e6fb-4417-bcaf-b38e62f8a2c5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.507300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.508277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.508564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.508877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] PATH_INFO: `/v3/projects/9352449cd0ff4e10891812531d9112f1/users/b8cc0771132c4f5598876cb2caf6121a/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.526485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9352449cd0ff4e10891812531d9112f1, user_id=b8cc0771132c4f5598876cb2caf6121a, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.529215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.555202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 933/1865] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:53 2022] PUT /identity/v3/projects/9352449cd0ff4e10891812531d9112f1/users/b8cc0771132c4f5598876cb2caf6121a/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:53.565967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544fd4e-fa14-4fb7-9ec8-44c0e4c98be1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.566252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544fd4e-fa14-4fb7-9ec8-44c0e4c98be1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.566525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544fd4e-fa14-4fb7-9ec8-44c0e4c98be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.595771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1544fd4e-fa14-4fb7-9ec8-44c0e4c98be1 None None] MFA Rules not processed for user `b8cc0771132c4f5598876cb2caf6121a`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:53.596995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1544fd4e-fa14-4fb7-9ec8-44c0e4c98be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:53.664098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 933/1866] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:53.668697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.731560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83b73-aab8-474c-ad00-64c249aab15f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.737667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.738275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.738557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.738819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.739452 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.741550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.772582 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 934/1867] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:53 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:53.773661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544fd4e-fa14-4fb7-9ec8-44c0e4c98be1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.784149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544fd4e-fa14-4fb7-9ec8-44c0e4c98be1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.839945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.840546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.840824 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.841096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.841729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.844113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.866161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 934/1868] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:53 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:53.870253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.881082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6d945-87d9-49c1-a712-6d5d08c19f64 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.886007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.886666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.886952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.887226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.887927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.889514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.900915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 935/1869] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:53.905119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.915886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60fb79-11a2-418d-b796-7e8f2b6d1ec3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:53.920637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:53.921367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:53.922305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:53.922584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] PATH_INFO: `/v3/projects/da5aead3e43d4b78ace6bed7fe14b960/users/70079e12352a4785bc0f8177bb16a6e5/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:53.939108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5aead3e43d4b78ace6bed7fe14b960, user_id=70079e12352a4785bc0f8177bb16a6e5, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:53.941765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:53.956533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 935/1870] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:53 2022] PUT /identity/v3/projects/da5aead3e43d4b78ace6bed7fe14b960/users/70079e12352a4785bc0f8177bb16a6e5/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:53.960560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:53.971077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cb4c6-787a-4714-9055-510ae1174ec9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:54.028286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d272f-1e08-4649-a0d1-75267c4833fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:54.028898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d272f-1e08-4649-a0d1-75267c4833fe admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:54.029176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d272f-1e08-4649-a0d1-75267c4833fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:54.029439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d272f-1e08-4649-a0d1-75267c4833fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:54.030078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d272f-1e08-4649-a0d1-75267c4833fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:54.031698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d272f-1e08-4649-a0d1-75267c4833fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:54.044950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 936/1871] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:54.046170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:54.056817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9be0bb-6eeb-43fb-8c67-a3ca258543dc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:54.061694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:54.062973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:54.063313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:54.063768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] PATH_INFO: `/v3/projects/da5aead3e43d4b78ace6bed7fe14b960/users/70079e12352a4785bc0f8177bb16a6e5/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:54.065873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5aead3e43d4b78ace6bed7fe14b960, user_id=70079e12352a4785bc0f8177bb16a6e5, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:54.068788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:54.072363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:54.083013 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 936/1872] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:54 2022] PUT /identity/v3/projects/da5aead3e43d4b78ace6bed7fe14b960/users/70079e12352a4785bc0f8177bb16a6e5/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:54.094313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b588c-8cd1-4fa6-a1e9-b565a2e2f6ba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:54.094817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b588c-8cd1-4fa6-a1e9-b565a2e2f6ba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:54.094926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b588c-8cd1-4fa6-a1e9-b565a2e2f6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:54.108696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:54.128072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ba5b588c-8cd1-4fa6-a1e9-b565a2e2f6ba None None] MFA Rules not processed for user `70079e12352a4785bc0f8177bb16a6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:54.129370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ba5b588c-8cd1-4fa6-a1e9-b565a2e2f6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:54.196187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 937/1873] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:54.207624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:54.209387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:54.268843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26671876-cfd5-4137-90cc-8d729a99e725 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:54.272769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:54.273344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:54.273732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:54.274009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:54.282931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:54.284383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:54.293387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 937/1874] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:54.369130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b588c-8cd1-4fa6-a1e9-b565a2e2f6ba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:54.379130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b588c-8cd1-4fa6-a1e9-b565a2e2f6ba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:54.382780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:54.383378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:54.383662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:54.383970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:54.445595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:54.447015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:54.455885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 938/1875] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:55.896080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:55.908062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a6e86-556d-4a73-a262-24dbe162e5e8 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:55.915403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783ada-c8d5-4083-9211-b763253ae405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:55.917724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783ada-c8d5-4083-9211-b763253ae405 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:55.917724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783ada-c8d5-4083-9211-b763253ae405 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:55.918203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783ada-c8d5-4083-9211-b763253ae405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:55.920407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-44783ada-c8d5-4083-9211-b763253ae405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:56.088630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783ada-c8d5-4083-9211-b763253ae405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:56.090679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783ada-c8d5-4083-9211-b763253ae405 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:56.105580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 938/1876] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:57.369542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:57.371040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:57.445180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3169-ba45-41ce-abea-2540a0e2c9ee service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:57.449609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:57.450241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:57.450520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:57.450791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] PATH_INFO: `/v3/users/6c40c138c61e41f78779739ea74dc4c5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:57.451939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c40c138c61e41f78779739ea74dc4c5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:57.453845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:57.493300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:57.513795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c40c138c61e41f78779739ea74dc4c5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.530108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c40c138c61e41f78779739ea74dc4c5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.548657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c40c138c61e41f78779739ea74dc4c5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.550113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c40c138c61e41f78779739ea74dc4c5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.562861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c40c138c61e41f78779739ea74dc4c5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.573272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 939/1877] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:57 2022] DELETE /identity/v3/users/6c40c138c61e41f78779739ea74dc4c5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:57.577857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:57.578386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:57.578386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:57.604849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:57.606309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:57.648934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:57.651331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:57.690500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 939/1878] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:57.702070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:57.705521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:57.766703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29338707-f9e1-4800-b48a-cac20eef10be admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:57.771669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:57.782126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:57.782817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:57.783095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:57.783371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] PATH_INFO: `/v3/users/b8cc0771132c4f5598876cb2caf6121a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:57.786539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8cc0771132c4f5598876cb2caf6121a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:57.788565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:57.801640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eb40f-e6d6-4a8c-8d17-c42b9679dcba None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:57.805700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b071c-44f9-4e25-a028-e96e210e6f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:57.806381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b071c-44f9-4e25-a028-e96e210e6f19 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:57.806670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b071c-44f9-4e25-a028-e96e210e6f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:57.806946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b071c-44f9-4e25-a028-e96e210e6f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:57.817900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b071c-44f9-4e25-a028-e96e210e6f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:57.819480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b071c-44f9-4e25-a028-e96e210e6f19 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:57.830091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 940/1879] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:57.861972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8cc0771132c4f5598876cb2caf6121a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.877704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8cc0771132c4f5598876cb2caf6121a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.885930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8cc0771132c4f5598876cb2caf6121a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.886172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8cc0771132c4f5598876cb2caf6121a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.895230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8cc0771132c4f5598876cb2caf6121a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:57.901863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 940/1880] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:57 2022] DELETE /identity/v3/users/b8cc0771132c4f5598876cb2caf6121a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:57.906655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:57.907207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:57.907529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:57.930934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:57.932211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:57.999410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 941/1881] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:58.010453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:58.011950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:58.049618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:58.086894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee0c32-4439-4464-86f2-d2a7778c7f9a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:58.090937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:58.091575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:58.091879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:58.092157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:58.101176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:58.102602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:58.114638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 941/1882] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:04:58.138675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:58.140076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:58.153554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b00eb-9fc4-40c0-97ef-5151bc384bd8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:58.159664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:58.160391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:58.161166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:58.161166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] PATH_INFO: `/v3/projects/128aacc4191a4d22a2138dc145acac1e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:58.162077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=128aacc4191a4d22a2138dc145acac1e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:58.164636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:58.220042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '128aacc4191a4d22a2138dc145acac1e', 'request_id': 'req-43116720-6dd6-4a5f-ac38-bd2d468b2565'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.227816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '128aacc4191a4d22a2138dc145acac1e', 'request_id': 'req-43116720-6dd6-4a5f-ac38-bd2d468b2565'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.235053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '128aacc4191a4d22a2138dc145acac1e', 'request_id': 'req-43116720-6dd6-4a5f-ac38-bd2d468b2565'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.253164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] The token cache is being invalidate because project 128aacc4191a4d22a2138dc145acac1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:58.253474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.254123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.258863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 942/1883] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:58 2022] DELETE /identity/v3/projects/128aacc4191a4d22a2138dc145acac1e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:58.260191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:58.270941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1624a4-3e85-4003-bdde-2f914c66f7f3 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:58.346427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:58.347201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:58.347547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:58.347975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] PATH_INFO: `/v3/users/7ad07c8d0a5d489ead604af553d2e733` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:58.349395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad07c8d0a5d489ead604af553d2e733)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:58.351624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:58.409179 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad07c8d0a5d489ead604af553d2e733'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.421745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad07c8d0a5d489ead604af553d2e733'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.430787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad07c8d0a5d489ead604af553d2e733'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.431448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad07c8d0a5d489ead604af553d2e733'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.439224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad07c8d0a5d489ead604af553d2e733'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.445831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 942/1884] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:58 2022] DELETE /identity/v3/users/7ad07c8d0a5d489ead604af553d2e733 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:58.545183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:58.546636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:58.563100 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43116720-6dd6-4a5f-ac38-bd2d468b2565 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:58.641401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:58.642662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:58.642884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:58.643160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] PATH_INFO: `/v3/projects/9352449cd0ff4e10891812531d9112f1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:58.644339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9352449cd0ff4e10891812531d9112f1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:58.646261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:58.648129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:58.711309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9352449cd0ff4e10891812531d9112f1', 'request_id': 'req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.717940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9352449cd0ff4e10891812531d9112f1', 'request_id': 'req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.727370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9352449cd0ff4e10891812531d9112f1', 'request_id': 'req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.753185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] The token cache is being invalidate because project 9352449cd0ff4e10891812531d9112f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:58.753185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.753543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.755059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 943/1885] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:58 2022] DELETE /identity/v3/projects/9352449cd0ff4e10891812531d9112f1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:58.759138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:58.760549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:58.784972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b63a-90de-4479-ab7c-06ff7dd0bbfd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:58.844855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:58.846578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:58.854324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:58.880366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:58.881025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:58.881307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:58.881657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] PATH_INFO: `/v3/users/70079e12352a4785bc0f8177bb16a6e5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:58.896702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70079e12352a4785bc0f8177bb16a6e5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:58.898923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:58.940818 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4e74-ddfa-4aed-8b44-974796f2ef65 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:58.946400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:58.947140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:58.947465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:58.947775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] PATH_INFO: `/v3/projects/f4de68e8cf9f494aaa9222b8a335f0cc` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:58.949662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4de68e8cf9f494aaa9222b8a335f0cc)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:58.951599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:58.972606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70079e12352a4785bc0f8177bb16a6e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.987717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70079e12352a4785bc0f8177bb16a6e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.995371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70079e12352a4785bc0f8177bb16a6e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:58.996182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70079e12352a4785bc0f8177bb16a6e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.002286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4de68e8cf9f494aaa9222b8a335f0cc', 'request_id': 'req-a563b7a2-4e15-41ab-a678-e0367b416b24'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.004089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-97f76bde-7724-4a32-b390-dd410194cc28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70079e12352a4785bc0f8177bb16a6e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.008194 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4de68e8cf9f494aaa9222b8a335f0cc', 'request_id': 'req-a563b7a2-4e15-41ab-a678-e0367b416b24'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.010166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 943/1886] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:04:58 2022] DELETE /identity/v3/users/70079e12352a4785bc0f8177bb16a6e5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:59.014682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4de68e8cf9f494aaa9222b8a335f0cc', 'request_id': 'req-a563b7a2-4e15-41ab-a678-e0367b416b24'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.028076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] The token cache is being invalidate because project f4de68e8cf9f494aaa9222b8a335f0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:59.028366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.029023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a563b7a2-4e15-41ab-a678-e0367b416b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.031161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 944/1887] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:58 2022] DELETE /identity/v3/projects/f4de68e8cf9f494aaa9222b8a335f0cc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:04:59.036361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909ea9-3fda-482c-8465-7e89b7fee0c0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.036593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909ea9-3fda-482c-8465-7e89b7fee0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.036874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909ea9-3fda-482c-8465-7e89b7fee0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.058243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-24909ea9-3fda-482c-8465-7e89b7fee0c0 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:59.059577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-24909ea9-3fda-482c-8465-7e89b7fee0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:59.131592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 944/1888] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:59.139460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b670-319c-4727-9016-fb5295e32c76 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.140127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b670-319c-4727-9016-fb5295e32c76 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.140127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b670-319c-4727-9016-fb5295e32c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.164259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3c37b670-319c-4727-9016-fb5295e32c76 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:59.165681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3c37b670-319c-4727-9016-fb5295e32c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:59.174245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 945/1889] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:04:59.178216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909ea9-3fda-482c-8465-7e89b7fee0c0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.188069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909ea9-3fda-482c-8465-7e89b7fee0c0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.193444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.193975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.194719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.194719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.195170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:59.195476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.197133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.209778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 945/1890] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:04:59.214088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37b670-319c-4727-9016-fb5295e32c76 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.223283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37b670-319c-4727-9016-fb5295e32c76 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.228007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.228601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.228883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.229155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.229788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.231714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.242893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:59.264353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 946/1891] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:59.265731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.276665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d13d3-59be-4c52-9ad1-44e239509cdc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.299044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.301128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:59.339338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:59.346071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.346698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.347467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.347985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.348683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.351921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.383668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 946/1892] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:59.385623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284c735-a1f1-46e7-af40-c0bcb1ee8441 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.387990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.391042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.392729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.393470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.395046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] PATH_INFO: `/v3/projects/da5aead3e43d4b78ace6bed7fe14b960` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.396739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5aead3e43d4b78ace6bed7fe14b960)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.403050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.405672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbeefeb-3710-447b-9a9c-9be7da9503ce admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.412180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.413136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.413537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.413818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.414864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.416497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.429677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 947/1893] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:59.434516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.447382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb56cb-a82f-4bb6-93fc-1735e7c8ec31 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.459206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5aead3e43d4b78ace6bed7fe14b960', 'request_id': 'req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.468898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5aead3e43d4b78ace6bed7fe14b960', 'request_id': 'req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.478003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5aead3e43d4b78ace6bed7fe14b960', 'request_id': 'req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.493233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] The token cache is being invalidate because project da5aead3e43d4b78ace6bed7fe14b960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:04:59.493789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.494494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d66a2ba7-63e8-4b29-bd99-5f400a4bc8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:04:59.496228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 947/1894] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:04:59 2022] DELETE /identity/v3/projects/da5aead3e43d4b78ace6bed7fe14b960 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:59.501061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b1817-2ef3-4dc4-b3b2-f44f3fdfb385 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.501943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b1817-2ef3-4dc4-b3b2-f44f3fdfb385 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.502525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b1817-2ef3-4dc4-b3b2-f44f3fdfb385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.522912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.522912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.522912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.522912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.523299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.523299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.523756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-464b1817-2ef3-4dc4-b3b2-f44f3fdfb385 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:59.525000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-464b1817-2ef3-4dc4-b3b2-f44f3fdfb385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:59.527713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 948/1895] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:59.531893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.533522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 948/1896] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:59.534385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:59.541021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7495f28-4ded-4e45-af3a-48d22ed685ca None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.541308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7495f28-4ded-4e45-af3a-48d22ed685ca None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.541684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7495f28-4ded-4e45-af3a-48d22ed685ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.548471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00e237-1634-4d98-84ea-6d41d908fc9d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.555051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5deca69-f48b-4a08-84c9-d84d7a3690cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.555688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deca69-f48b-4a08-84c9-d84d7a3690cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.556019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deca69-f48b-4a08-84c9-d84d7a3690cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.556320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deca69-f48b-4a08-84c9-d84d7a3690cd admin admin] PATH_INFO: `/v3/projects/c863de9cc7234d29a7301d583a3faa0e/users/cbcf98673ffe41aa8e4bf7a9f4ae6ab9/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.562546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f7495f28-4ded-4e45-af3a-48d22ed685ca None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:59.564342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f7495f28-4ded-4e45-af3a-48d22ed685ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:59.575942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 949/1897] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:59.576099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5deca69-f48b-4a08-84c9-d84d7a3690cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c863de9cc7234d29a7301d583a3faa0e, user_id=cbcf98673ffe41aa8e4bf7a9f4ae6ab9, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.577277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7495f28-4ded-4e45-af3a-48d22ed685ca None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.577724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5deca69-f48b-4a08-84c9-d84d7a3690cd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.587797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7495f28-4ded-4e45-af3a-48d22ed685ca None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.593757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 949/1898] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:59 2022] PUT /identity/v3/projects/c863de9cc7234d29a7301d583a3faa0e/users/cbcf98673ffe41aa8e4bf7a9f4ae6ab9/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:04:59.605794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724696bc-2c64-4b4c-aba7-a575d204f03f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.606078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724696bc-2c64-4b4c-aba7-a575d204f03f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.606336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724696bc-2c64-4b4c-aba7-a575d204f03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.636650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-724696bc-2c64-4b4c-aba7-a575d204f03f None None] MFA Rules not processed for user `cbcf98673ffe41aa8e4bf7a9f4ae6ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:04:59.637884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-724696bc-2c64-4b4c-aba7-a575d204f03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:04:59.659655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.660431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.661153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.661433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.662080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:04:59.662690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.664365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.677982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 950/1899] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:04:59.683453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.689634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:59.695423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd06a8-b17f-42a7-8944-84b23be235d0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.701189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.701798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.702260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.702552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.704054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.706803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.713822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 950/1900] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:04:59.718243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724696bc-2c64-4b4c-aba7-a575d204f03f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.731363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724696bc-2c64-4b4c-aba7-a575d204f03f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.737309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.737799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.738087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.738364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.738548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 951/1901] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:59.738998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.740944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.757046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.778539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 951/1902] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:59.783263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.804230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:04:59.835541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1f447-b89a-40f0-8e4f-00a81f1d1d39 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.858610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2ffa5-8a6f-4f1a-83da-1beb8302341a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.865189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.865975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.866257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.866532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.867236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.869226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.895454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 952/1903] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:04:59.899984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.905696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.906370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.906592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.906862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.908829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.910698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.916643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a2130-4a98-4742-adf4-bb668d004939 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.936085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.936085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.936085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.936085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.936627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.936627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.940694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 952/1904] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:04:59 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:04:59.944232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 953/1905] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:04:59.944786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.945074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.954984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3282b2d-6ee6-4816-97ec-dacc8bcb4968 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.955234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817246b6-bd3c-4244-8a39-b8cd3ce21890 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.960104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.960233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.960700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.960808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.960986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.961076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.961262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.961351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] PATH_INFO: `/v3/projects/7bf614d9db9d4e54bf662340ebba47d2/users/42fa69ef4a874e2c9b03b4095bdf240d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.961931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.963518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.975525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 954/1906] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:04:59.979240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:04:59.980452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf614d9db9d4e54bf662340ebba47d2, user_id=42fa69ef4a874e2c9b03b4095bdf240d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:04:59.983150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:04:59.991392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe559d-6eda-43c0-af13-432ef9bce4e8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:04:59.996956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:04:59.997571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:04:59.997854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:04:59.998132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:04:59.998777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.000418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.004024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 953/1907] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:04:59 2022] PUT /identity/v3/projects/7bf614d9db9d4e54bf662340ebba47d2/users/42fa69ef4a874e2c9b03b4095bdf240d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:00.005060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.012388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 955/1908] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:00.016478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.026865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e462b9-4f6c-427f-8e85-e4927e22891c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.064986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:00.082595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f6de-0b4c-45a0-89e1-f6f77bbcef57 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.088052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.088662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.088949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.089226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.089873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.091474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.094663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43171e3f-807d-4595-b050-8ca64647f2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.096155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43171e3f-807d-4595-b050-8ca64647f2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.096668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43171e3f-807d-4595-b050-8ca64647f2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.097166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43171e3f-807d-4595-b050-8ca64647f2a4 admin admin] PATH_INFO: `/v3/projects/890b4586db834e268dc70b88100a3fc9/users/b093e8657dba41e6998cea3ba7667f57/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.103878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 954/1909] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:00.108418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.117726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43171e3f-807d-4595-b050-8ca64647f2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=890b4586db834e268dc70b88100a3fc9, user_id=b093e8657dba41e6998cea3ba7667f57, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.119524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800728c-a1d7-47ee-8c00-2e7e3056e947 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.121441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43171e3f-807d-4595-b050-8ca64647f2a4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.124309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476b63-07d9-44f9-bc36-401d6935ea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.124961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476b63-07d9-44f9-bc36-401d6935ea1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.125252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476b63-07d9-44f9-bc36-401d6935ea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.125532 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476b63-07d9-44f9-bc36-401d6935ea1b admin admin] PATH_INFO: `/v3/projects/7bf614d9db9d4e54bf662340ebba47d2/users/42fa69ef4a874e2c9b03b4095bdf240d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.132341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa476b63-07d9-44f9-bc36-401d6935ea1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf614d9db9d4e54bf662340ebba47d2, user_id=42fa69ef4a874e2c9b03b4095bdf240d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.135588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa476b63-07d9-44f9-bc36-401d6935ea1b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.140866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 956/1910] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:00 2022] PUT /identity/v3/projects/890b4586db834e268dc70b88100a3fc9/users/b093e8657dba41e6998cea3ba7667f57/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:00.152987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e6a75-497e-41da-9ae7-5af2921a296e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.153279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e6a75-497e-41da-9ae7-5af2921a296e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.153561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e6a75-497e-41da-9ae7-5af2921a296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.157639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 955/1911] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:00 2022] PUT /identity/v3/projects/7bf614d9db9d4e54bf662340ebba47d2/users/42fa69ef4a874e2c9b03b4095bdf240d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:00.168048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.168542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.168822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.191701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-838e6a75-497e-41da-9ae7-5af2921a296e None None] MFA Rules not processed for user `b093e8657dba41e6998cea3ba7667f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:00.192971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-838e6a75-497e-41da-9ae7-5af2921a296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:00.197171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] MFA Rules not processed for user `42fa69ef4a874e2c9b03b4095bdf240d`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:00.198356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:00.239516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:00.261791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 957/1912] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:00.266199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e6a75-497e-41da-9ae7-5af2921a296e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.267426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 956/1913] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:00.278226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.279608 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:00.332811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e6a75-497e-41da-9ae7-5af2921a296e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.336274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1aa5f-a028-4f11-8112-1dc555925ce2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.337360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.337970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.338250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.338523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.339157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.339930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.340539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.340805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.341072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.341163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.350080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.351499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.361658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 957/1914] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:00.371599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 958/1915] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:00 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:00.375522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.386736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87908f52-fde8-400e-942f-b636c1eb6dd6 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.451877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.452488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.452767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.453039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.453825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.455579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.465523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:00.485314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 958/1916] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:00 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:00.487306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.498307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee25a6-9f43-4d56-b5ba-f69a07866ddc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.504759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.504960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.505145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.505418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.506063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.507647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.518634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 959/1917] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:00.522884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.534345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3c687-f894-4b75-95ca-2c08dc03c1b5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.540002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.540884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.541357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.541760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] PATH_INFO: `/v3/projects/ad1325fc5afa438bad986c936a11eb8b/users/c9e0ffd2d9764d13b0dc4c0e58850d56/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.559344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1325fc5afa438bad986c936a11eb8b, user_id=c9e0ffd2d9764d13b0dc4c0e58850d56, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.562380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.578208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 959/1918] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:00 2022] PUT /identity/v3/projects/ad1325fc5afa438bad986c936a11eb8b/users/c9e0ffd2d9764d13b0dc4c0e58850d56/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:00.582323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.592796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b8de0-7995-4786-9977-da062dba3b4c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.663573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93489546-5055-4d8e-b9eb-016bbba99496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.663715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489546-5055-4d8e-b9eb-016bbba99496 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.664276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489546-5055-4d8e-b9eb-016bbba99496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.664556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489546-5055-4d8e-b9eb-016bbba99496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.665892 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93489546-5055-4d8e-b9eb-016bbba99496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.666984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93489546-5055-4d8e-b9eb-016bbba99496 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.678615 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 960/1919] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:00.682841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.694933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3736-a4dd-4ff5-93eb-878a37c43444 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.700129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.701125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.701285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.701667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] PATH_INFO: `/v3/projects/ad1325fc5afa438bad986c936a11eb8b/users/c9e0ffd2d9764d13b0dc4c0e58850d56/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.703697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1325fc5afa438bad986c936a11eb8b, user_id=c9e0ffd2d9764d13b0dc4c0e58850d56, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.707814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.719477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 960/1920] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:00 2022] PUT /identity/v3/projects/ad1325fc5afa438bad986c936a11eb8b/users/c9e0ffd2d9764d13b0dc4c0e58850d56/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:00.731029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56041-dd63-422f-aa81-792832b39637 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.731392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56041-dd63-422f-aa81-792832b39637 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.731730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56041-dd63-422f-aa81-792832b39637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.763244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-70d56041-dd63-422f-aa81-792832b39637 None None] MFA Rules not processed for user `c9e0ffd2d9764d13b0dc4c0e58850d56`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:00.764143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-70d56041-dd63-422f-aa81-792832b39637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:00.817019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:00.849743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 961/1921] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:00.862231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:00.945458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3adf8-e352-48b5-9915-4d504549826c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:00.949654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b43db2-90b1-4f62-a97d-08b0d9adbc74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:00.950569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43db2-90b1-4f62-a97d-08b0d9adbc74 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:00.951041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43db2-90b1-4f62-a97d-08b0d9adbc74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:00.951567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43db2-90b1-4f62-a97d-08b0d9adbc74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:00.965581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b43db2-90b1-4f62-a97d-08b0d9adbc74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:00.967077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b43db2-90b1-4f62-a97d-08b0d9adbc74 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:00.981849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 961/1922] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:01.014651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d56041-dd63-422f-aa81-792832b39637 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:01.105843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d56041-dd63-422f-aa81-792832b39637 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:01.113164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:01.113981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:01.114319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:01.114655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] PATH_INFO: `/v3/users/b093e8657dba41e6998cea3ba7667f57` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:01.115898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b093e8657dba41e6998cea3ba7667f57)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:01.117956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:01.171818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b093e8657dba41e6998cea3ba7667f57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.196282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b093e8657dba41e6998cea3ba7667f57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.205635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b093e8657dba41e6998cea3ba7667f57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.206571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b093e8657dba41e6998cea3ba7667f57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.218519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b093e8657dba41e6998cea3ba7667f57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.230269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 962/1923] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:01 2022] DELETE /identity/v3/users/b093e8657dba41e6998cea3ba7667f57 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:01.234630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:01.234920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:01.235199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:01.263944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:01.265377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:01.356052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 962/1924] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:01.373899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:01.377542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:01.444544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444d32c-8569-4400-8beb-da6ce84bada8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:01.451651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:01.452327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:01.452613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:01.452891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:01.463394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:01.464865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:01.478047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 963/1925] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:01.780440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:01.783064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:01.804152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f5972-bd48-4e3d-a7fb-de728051dab3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:01.811630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:01.816062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:01.816500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:01.816985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] PATH_INFO: `/v3/projects/890b4586db834e268dc70b88100a3fc9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:01.818310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] RBAC: Authorizing `identity:delete_project(project_id=890b4586db834e268dc70b88100a3fc9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:01.820708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:01.883184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890b4586db834e268dc70b88100a3fc9', 'request_id': 'req-53486be8-8dc8-4d22-9837-79ba2c44eacf'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.889647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:01.893664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890b4586db834e268dc70b88100a3fc9', 'request_id': 'req-53486be8-8dc8-4d22-9837-79ba2c44eacf'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.908647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890b4586db834e268dc70b88100a3fc9', 'request_id': 'req-53486be8-8dc8-4d22-9837-79ba2c44eacf'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.926869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] The token cache is being invalidate because project 890b4586db834e268dc70b88100a3fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:01.927216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.927973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:01.929655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 963/1926] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:01 2022] DELETE /identity/v3/projects/890b4586db834e268dc70b88100a3fc9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:01.934209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:01.940784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:01.948165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14143cd-b212-47b5-9b98-bfc6496f1ba8 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:01.993058 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:01.994606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:02.025443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:02.026089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:02.026640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:02.026640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] PATH_INFO: `/v3/users/c9e0ffd2d9764d13b0dc4c0e58850d56` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:02.027865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e0ffd2d9764d13b0dc4c0e58850d56)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:02.034360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:02.087243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53486be8-8dc8-4d22-9837-79ba2c44eacf admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:02.093671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:02.094343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:02.095424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:02.095925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] PATH_INFO: `/v3/users/cbcf98673ffe41aa8e4bf7a9f4ae6ab9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:02.097705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbcf98673ffe41aa8e4bf7a9f4ae6ab9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:02.100093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:02.137102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e0ffd2d9764d13b0dc4c0e58850d56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.157258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e0ffd2d9764d13b0dc4c0e58850d56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.171554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e0ffd2d9764d13b0dc4c0e58850d56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.173114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e0ffd2d9764d13b0dc4c0e58850d56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.183896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-256909dc-1ea3-4b14-866b-c51ffb78e6ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e0ffd2d9764d13b0dc4c0e58850d56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.187434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbcf98673ffe41aa8e4bf7a9f4ae6ab9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.194739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 964/1927] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:01 2022] DELETE /identity/v3/users/c9e0ffd2d9764d13b0dc4c0e58850d56 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:02.205936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbcf98673ffe41aa8e4bf7a9f4ae6ab9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.207271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbcf98673ffe41aa8e4bf7a9f4ae6ab9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.217912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbcf98673ffe41aa8e4bf7a9f4ae6ab9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.218979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbcf98673ffe41aa8e4bf7a9f4ae6ab9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.239585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 964/1928] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:01 2022] DELETE /identity/v3/users/cbcf98673ffe41aa8e4bf7a9f4ae6ab9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:02.244836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:02.245135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:02.245417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:02.273517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:02.274940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:02.364542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 965/1929] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:02.374467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:02.376053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:02.502120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec3cfc-3cfb-4c47-a730-7c4737e721c3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:02.506875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee154b94-f369-4a45-af4e-b26b36ef16ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:02.507510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154b94-f369-4a45-af4e-b26b36ef16ab service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:02.507788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154b94-f369-4a45-af4e-b26b36ef16ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:02.508094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154b94-f369-4a45-af4e-b26b36ef16ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:02.523605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee154b94-f369-4a45-af4e-b26b36ef16ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:02.525309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee154b94-f369-4a45-af4e-b26b36ef16ab service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:02.545858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 965/1930] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:02.668492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:02.669991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:02.688004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95b2e7-f50c-400b-8ab2-a296081ad5c5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:02.704757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:02.705490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:02.705490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:02.705749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] PATH_INFO: `/v3/projects/ad1325fc5afa438bad986c936a11eb8b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:02.707803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1325fc5afa438bad986c936a11eb8b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:02.710146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:02.737451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:02.784128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1325fc5afa438bad986c936a11eb8b', 'request_id': 'req-63585826-a0d6-4e30-b26e-4280daecf28d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.791054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1325fc5afa438bad986c936a11eb8b', 'request_id': 'req-63585826-a0d6-4e30-b26e-4280daecf28d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.799405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1325fc5afa438bad986c936a11eb8b', 'request_id': 'req-63585826-a0d6-4e30-b26e-4280daecf28d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.813158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] The token cache is being invalidate because project ad1325fc5afa438bad986c936a11eb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:02.813474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.814115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63585826-a0d6-4e30-b26e-4280daecf28d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:02.816177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 966/1931] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:02 2022] DELETE /identity/v3/projects/ad1325fc5afa438bad986c936a11eb8b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:02.820661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:02.821139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:02.821215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:02.831750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:02.851485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:02.853531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:02.956930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 966/1932] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:02.965077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a10a2-8708-42e9-bcaf-63c267ed039d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:02.965590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a10a2-8708-42e9-bcaf-63c267ed039d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:02.965590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a10a2-8708-42e9-bcaf-63c267ed039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:02.980814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:02.982436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:03.001628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e16a10a2-8708-42e9-bcaf-63c267ed039d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:03.003511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e16a10a2-8708-42e9-bcaf-63c267ed039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:03.011429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb23455-2aad-4a89-9506-8f5b7ce776aa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.020191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.020844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.021118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.021395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] PATH_INFO: `/v3/projects/c863de9cc7234d29a7301d583a3faa0e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.025838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 967/1933] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:03.025940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c863de9cc7234d29a7301d583a3faa0e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.025940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.026775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a10a2-8708-42e9-bcaf-63c267ed039d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.043955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a10a2-8708-42e9-bcaf-63c267ed039d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.052191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.053277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.054194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.054619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.055969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:03.056480 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.059380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.081867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 968/1934] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:05:03.083311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.087780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c863de9cc7234d29a7301d583a3faa0e', 'request_id': 'req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.094832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd420a6e-ad46-486a-995d-b29a387d8468 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.097685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c863de9cc7234d29a7301d583a3faa0e', 'request_id': 'req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.104579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c863de9cc7234d29a7301d583a3faa0e', 'request_id': 'req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.122178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] The token cache is being invalidate because project c863de9cc7234d29a7301d583a3faa0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:03.122610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.124042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.127419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 967/1935] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:02 2022] DELETE /identity/v3/projects/c863de9cc7234d29a7301d583a3faa0e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:03.130995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.133507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:03.162306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.163015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.163296 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.163573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.164377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.166433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.200041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 969/1936] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:03 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:03.203405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.206426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:03.221529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8d19-49e8-4bf9-80e6-ed954ff5e710 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.260655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:03.276030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee5eae-878a-4080-87e9-8fbf41454e09 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.280758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.281427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.281641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.281921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.282692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.284924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.287052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.287680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.287989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.288265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] PATH_INFO: `/v3/users/42fa69ef4a874e2c9b03b4095bdf240d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.289398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fa69ef4a874e2c9b03b4095bdf240d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.291294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.302836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:03.317698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 970/1937] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:03 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:03.319860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.331666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb67b3-b35c-449b-aa77-85a612e7c2b1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.338035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.338644 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.338922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.339201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.339861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.341469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.352866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 971/1938] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:03.357676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.359187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fa69ef4a874e2c9b03b4095bdf240d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.370672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab699-f316-48cb-929d-5a8c06f8aa6f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.376921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.377452 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.377894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.378244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.381094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.381094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.395849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 972/1939] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:03.396871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.405571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fa69ef4a874e2c9b03b4095bdf240d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.406834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fa69ef4a874e2c9b03b4095bdf240d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.412247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfacc5d-9bea-4436-a84b-3f9ce89216fc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.416592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fa69ef4a874e2c9b03b4095bdf240d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.417274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d5c90747-d4f5-416c-a8b9-68565fe2b7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fa69ef4a874e2c9b03b4095bdf240d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:03.417454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.418151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.418434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.418713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] PATH_INFO: `/v3/projects/517c8a2bd2804038a7fd188d6a6ac993/users/6225b3556ecb466ab6945665dbbb172e/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.435943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 968/1940] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:03 2022] DELETE /identity/v3/users/42fa69ef4a874e2c9b03b4095bdf240d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:03.440596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] RBAC: Authorizing `identity:create_grant(project_id=517c8a2bd2804038a7fd188d6a6ac993, user_id=6225b3556ecb466ab6945665dbbb172e, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.448901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.468562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 973/1941] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:03 2022] PUT /identity/v3/projects/517c8a2bd2804038a7fd188d6a6ac993/users/6225b3556ecb466ab6945665dbbb172e/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:03.479035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29a41-ccbb-4f5b-91e5-3324b5c72b5f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.479358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29a41-ccbb-4f5b-91e5-3324b5c72b5f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.479624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29a41-ccbb-4f5b-91e5-3324b5c72b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.535358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e4f29a41-ccbb-4f5b-91e5-3324b5c72b5f None None] MFA Rules not processed for user `6225b3556ecb466ab6945665dbbb172e`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:03.537056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e4f29a41-ccbb-4f5b-91e5-3324b5c72b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:03.643913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 969/1942] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:03.646515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.650489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:03.855317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b2a7f-46dd-4a11-bb7a-24d08c29026b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.861004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:03.862147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:03.862426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:03.862849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:03.863511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:03.865675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:03.935532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 974/1943] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:03 2022] POST /identity/v3/projects => generated 381 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:03.936764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29a41-ccbb-4f5b-91e5-3324b5c72b5f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.949009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29a41-ccbb-4f5b-91e5-3324b5c72b5f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:03.987672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:03.989676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:04.008672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:04.031544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.032236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.032875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.032875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.033498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.036903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.066756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 970/1944] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:03 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:04.071091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.083956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267da9cc-a4b0-438b-af9a-8c16069f3dac admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.096903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.097036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad411a-1f7c-42ff-b319-96141637b76a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.097614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.097903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.098175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.099008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.101690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.103568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.104390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.104725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.105006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] PATH_INFO: `/v3/projects/7bf614d9db9d4e54bf662340ebba47d2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.106425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf614d9db9d4e54bf662340ebba47d2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.108474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.119567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 971/1945] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:04.123074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.135679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c137-46cf-4009-a044-296a7ccaef1c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.140848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.141538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.142504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.142821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] PATH_INFO: `/v3/projects/04c939821b324a4e883ffc38ac2f0a0b/users/5b423e97418f474e99aa5b105bc6b3e6/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.159269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c939821b324a4e883ffc38ac2f0a0b, user_id=5b423e97418f474e99aa5b105bc6b3e6, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.161953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.162426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:04.171065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf614d9db9d4e54bf662340ebba47d2', 'request_id': 'req-cec7c18c-fc25-43d0-81e2-517fc3666f56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:04.175907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 972/1946] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:04 2022] PUT /identity/v3/projects/04c939821b324a4e883ffc38ac2f0a0b/users/5b423e97418f474e99aa5b105bc6b3e6/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:04.177432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf614d9db9d4e54bf662340ebba47d2', 'request_id': 'req-cec7c18c-fc25-43d0-81e2-517fc3666f56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:04.179520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.184165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf614d9db9d4e54bf662340ebba47d2', 'request_id': 'req-cec7c18c-fc25-43d0-81e2-517fc3666f56'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:04.197031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] The token cache is being invalidate because project 7bf614d9db9d4e54bf662340ebba47d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:04.197227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:04.197878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cec7c18c-fc25-43d0-81e2-517fc3666f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:04.199453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 975/1947] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:03 2022] DELETE /identity/v3/projects/7bf614d9db9d4e54bf662340ebba47d2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:04.205052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f9fb7-a8f5-43bc-8394-a15214de7e9e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.205346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f9fb7-a8f5-43bc-8394-a15214de7e9e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.205612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f9fb7-a8f5-43bc-8394-a15214de7e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.226226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f68f9fb7-a8f5-43bc-8394-a15214de7e9e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:04.227432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f68f9fb7-a8f5-43bc-8394-a15214de7e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:04.242215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a832-d8c5-4dbf-98c0-f6e56c0f341f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.249288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.249967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.250352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.250525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.251600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.254127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.270089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 973/1948] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:04.271711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.274206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:04.289333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fced87-a97c-4362-8f52-eae7ef1ae0ee admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.293654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 976/1949] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:04.294564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec32b82-69b9-4754-a34b-44249f766f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.295228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32b82-69b9-4754-a34b-44249f766f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.295505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32b82-69b9-4754-a34b-44249f766f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.295774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32b82-69b9-4754-a34b-44249f766f0e admin admin] PATH_INFO: `/v3/projects/04c939821b324a4e883ffc38ac2f0a0b/users/5b423e97418f474e99aa5b105bc6b3e6/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.297727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32b82-69b9-4754-a34b-44249f766f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c939821b324a4e883ffc38ac2f0a0b, user_id=5b423e97418f474e99aa5b105bc6b3e6, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.301152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32b82-69b9-4754-a34b-44249f766f0e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.301997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a05d3-5d98-46ff-9076-cf6c4086d086 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.302405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a05d3-5d98-46ff-9076-cf6c4086d086 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.302680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a05d3-5d98-46ff-9076-cf6c4086d086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.313074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 974/1950] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:04 2022] PUT /identity/v3/projects/04c939821b324a4e883ffc38ac2f0a0b/users/5b423e97418f474e99aa5b105bc6b3e6/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:04.324926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.325204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.325477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.327321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e70a05d3-5d98-46ff-9076-cf6c4086d086 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:04.327825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e70a05d3-5d98-46ff-9076-cf6c4086d086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:04.364491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] MFA Rules not processed for user `5b423e97418f474e99aa5b105bc6b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:04.366180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:04.401403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 977/1951] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:04.406149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a05d3-5d98-46ff-9076-cf6c4086d086 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.415416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a05d3-5d98-46ff-9076-cf6c4086d086 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.420049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.420678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.420954 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.421225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.421883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:04.422187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.424064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.435606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 978/1952] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:04.440712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.450303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d306f-2bc3-4cff-8bc4-4c933a604918 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.455578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.456200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.456484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.456759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.457376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.459273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.463031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 975/1953] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:04.475562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.476996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:04.486738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:04.490789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 979/1954] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:04 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:04.491780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.534441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:04.539364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b04bf-6886-4700-a059-5897820a1d48 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.563283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250ccef-035f-4c6c-8a90-95fbcce4d90b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.569300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.569911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.570198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.570479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.571129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.573083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.598869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 980/1955] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:04 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:04.602914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.606063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a21af4-19c5-4172-bf4d-d2efe5a95db3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.607017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a21af4-19c5-4172-bf4d-d2efe5a95db3 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.607496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a21af4-19c5-4172-bf4d-d2efe5a95db3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.607770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a21af4-19c5-4172-bf4d-d2efe5a95db3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.613792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9b112-3b16-411a-85fe-a63dcaf1414d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.619271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.619775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.620086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.620357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.621000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.622576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.636622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 981/1956] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:04.638263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.650240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00f868-c15d-494d-87f3-674c6e9f3456 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.655435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.656051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.656459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.656737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.657427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.659031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.670338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 982/1957] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:04.675132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.679303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a21af4-19c5-4172-bf4d-d2efe5a95db3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.681093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a21af4-19c5-4172-bf4d-d2efe5a95db3 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.687439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9f549-d796-438e-897b-853006d937c3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.692687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 976/1958] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:04.694408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.695144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.695530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.696032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] PATH_INFO: `/v3/projects/fc6e235982a24981b5997b8bf42d59b2/users/6c2e5626258a4f62a3004680eb93904f/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.715690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6e235982a24981b5997b8bf42d59b2, user_id=6c2e5626258a4f62a3004680eb93904f, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.718427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.733011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 983/1959] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:04 2022] PUT /identity/v3/projects/fc6e235982a24981b5997b8bf42d59b2/users/6c2e5626258a4f62a3004680eb93904f/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:04.745210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dde21-b2e9-4505-a68e-b4c36d7fbf34 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.745498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dde21-b2e9-4505-a68e-b4c36d7fbf34 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.745778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dde21-b2e9-4505-a68e-b4c36d7fbf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.775309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b89dde21-b2e9-4505-a68e-b4c36d7fbf34 None None] MFA Rules not processed for user `6c2e5626258a4f62a3004680eb93904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:04.776552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b89dde21-b2e9-4505-a68e-b4c36d7fbf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:04.844595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 977/1960] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:04.849288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:04.935559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec545b48-e7b4-4779-8dc9-45e61a18b14b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:04.953438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:04.953438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:04.953746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:04.954836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:04.955612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:04.958113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:04.999930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 984/1961] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:04 2022] POST /identity/v3/projects => generated 399 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:05.001836 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dde21-b2e9-4505-a68e-b4c36d7fbf34 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.009008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:05.024653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dde21-b2e9-4505-a68e-b4c36d7fbf34 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.097924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.111933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.113056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.113508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.113936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.115019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.117877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.158175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 978/1962] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:05 2022] POST /identity/v3/users => generated 306 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:05.165334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.184491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e223-7bb4-47e3-aec2-d835e5b4c4b7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.191334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.192052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.192299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.192580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.193303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.195044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.195482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa2271-8e6e-4ceb-8da1-dace7bd41ab5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.201032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b423e97418f474e99aa5b105bc6b3e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04c939821b324a4e883ffc38ac2f0a0b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.201833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.202126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.202406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.203058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:05.203368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.205948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.207540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 979/1963] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:05.211474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.216477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:05.223378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe1ab4-f74e-4962-8a6b-48663f7e16ac admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.224636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 985/1964] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:05.228780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.233494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.234257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.234562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.234958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] PATH_INFO: `/v3/projects/8b2afc78752d4170b555e0f1a3b43786/users/a9895ba13a4e4c01a6fee986f29cb74b/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.241717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe202029-1b4d-4847-a4ef-b1e18cb350eb tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.246939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b423e97418f474e99aa5b105bc6b3e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04c939821b324a4e883ffc38ac2f0a0b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.247907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.248236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.248600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.249219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.251822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.254603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2afc78752d4170b555e0f1a3b43786, user_id=a9895ba13a4e4c01a6fee986f29cb74b, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.257833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.275933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 980/1965] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:05 2022] PUT /identity/v3/projects/8b2afc78752d4170b555e0f1a3b43786/users/a9895ba13a4e4c01a6fee986f29cb74b/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:05.278870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.283856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 986/1966] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:05 2022] POST /identity/v3/projects => generated 354 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:05.287403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.357078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050de9bc-6046-408a-b3d9-757241115681 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.378015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9421e-044e-487b-ba77-ca57fb22edb1 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.384241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b423e97418f474e99aa5b105bc6b3e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04c939821b324a4e883ffc38ac2f0a0b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.384888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.385456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.385456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.386128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:05.386438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.388211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.407915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 987/1967] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:05.421732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.422344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.422625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.423522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.424307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.425969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.431582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:05.437377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 981/1968] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:05.441511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.452924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.456108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd5a9d-c502-446d-bb0b-c9b252e6458b tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.461121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270084b0-33b0-464c-a547-7dc56a5ff54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.461791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270084b0-33b0-464c-a547-7dc56a5ff54e admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.462072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270084b0-33b0-464c-a547-7dc56a5ff54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.462350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270084b0-33b0-464c-a547-7dc56a5ff54e admin admin] PATH_INFO: `/v3/projects/8b2afc78752d4170b555e0f1a3b43786/users/a9895ba13a4e4c01a6fee986f29cb74b/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.464306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270084b0-33b0-464c-a547-7dc56a5ff54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2afc78752d4170b555e0f1a3b43786, user_id=a9895ba13a4e4c01a6fee986f29cb74b, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.465350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34984109-e211-4ac0-9cd2-3cefcc2cb68f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.466963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270084b0-33b0-464c-a547-7dc56a5ff54e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.471961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b423e97418f474e99aa5b105bc6b3e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04c939821b324a4e883ffc38ac2f0a0b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.472594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.472871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.473146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] PATH_INFO: `/v3/projects/c93a1baad89249aab1961b68f84b1867` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.474259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c93a1baad89249aab1961b68f84b1867)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.476351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.478985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 988/1969] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:05 2022] PUT /identity/v3/projects/8b2afc78752d4170b555e0f1a3b43786/users/a9895ba13a4e4c01a6fee986f29cb74b/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:05.489983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.490589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.490989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.524967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] MFA Rules not processed for user `a9895ba13a4e4c01a6fee986f29cb74b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:05.526356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:05.527684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c93a1baad89249aab1961b68f84b1867', 'request_id': 'req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.536798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c93a1baad89249aab1961b68f84b1867', 'request_id': 'req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.544843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c93a1baad89249aab1961b68f84b1867', 'request_id': 'req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.559716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] The token cache is being invalidate because project c93a1baad89249aab1961b68f84b1867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:05.560143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.560836 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.562384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 982/1970] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:05 2022] DELETE /identity/v3/projects/c93a1baad89249aab1961b68f84b1867 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:05.612396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 989/1971] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:05.623505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.624976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:05.685476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5647e-73dd-4cc5-86a5-d10f754a18d3 tempest-VolumeQuotaClassesTest-1208555935 tempest-VolumeQuotaClassesTest-1208555935-project-admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.689192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.689799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.690086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.690360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.702490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.702490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.711590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 983/1972] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:05.784601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.786067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:05.813829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:05.815373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:05.826641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1828b5-0b93-494a-b572-1eb39aef90c9 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.831060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2fce5-1feb-4305-9a05-10673b45723b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.831705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2fce5-1feb-4305-9a05-10673b45723b service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.832011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2fce5-1feb-4305-9a05-10673b45723b service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.832286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2fce5-1feb-4305-9a05-10673b45723b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.834070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-78b2fce5-1feb-4305-9a05-10673b45723b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:05.839482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:05.862196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c75dc-8ae9-4854-9dea-f6137724cece None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:05.867278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:05.868405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.868405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.868631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] PATH_INFO: `/v3/users/6225b3556ecb466ab6945665dbbb172e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.869771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6225b3556ecb466ab6945665dbbb172e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.871551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.903565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2fce5-1feb-4305-9a05-10673b45723b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:05.905423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2fce5-1feb-4305-9a05-10673b45723b service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:05.916578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 984/1973] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:05.931431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6225b3556ecb466ab6945665dbbb172e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.943327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6225b3556ecb466ab6945665dbbb172e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.950848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6225b3556ecb466ab6945665dbbb172e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.951471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6225b3556ecb466ab6945665dbbb172e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.958898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6225b3556ecb466ab6945665dbbb172e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:05.964849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 990/1974] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:05 2022] DELETE /identity/v3/users/6225b3556ecb466ab6945665dbbb172e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:05.969302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:05.969592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:05.970563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:05.990378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:05.991586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:06.011055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:06.059544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 985/1975] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:06.070618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:06.072345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:06.140825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b445c9a-8796-406c-b724-582587d57476 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:06.145174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:06.146152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:06.146584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:06.147026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:06.161834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:06.164953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:06.177063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 991/1976] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:06.541021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:06.542515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:06.555448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d494e-46ee-4ce9-9e2e-067659f3aa15 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:06.560541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:06.561177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:06.561461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:06.561745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] PATH_INFO: `/v3/projects/517c8a2bd2804038a7fd188d6a6ac993` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:06.562864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517c8a2bd2804038a7fd188d6a6ac993)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:06.565384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:06.613836 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517c8a2bd2804038a7fd188d6a6ac993', 'request_id': 'req-9f379b11-1423-457b-b573-67b39ef5e3a6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.619334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517c8a2bd2804038a7fd188d6a6ac993', 'request_id': 'req-9f379b11-1423-457b-b573-67b39ef5e3a6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.625597 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517c8a2bd2804038a7fd188d6a6ac993', 'request_id': 'req-9f379b11-1423-457b-b573-67b39ef5e3a6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.638188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] The token cache is being invalidate because project 517c8a2bd2804038a7fd188d6a6ac993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:06.638484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.639114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.640715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 986/1977] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:06 2022] DELETE /identity/v3/projects/517c8a2bd2804038a7fd188d6a6ac993 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:06.645007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:06.654926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1c3a-49ff-45ea-bdfd-08ca27834928 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:06.711721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:06.712388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:06.712667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:06.712937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] PATH_INFO: `/v3/users/5b423e97418f474e99aa5b105bc6b3e6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:06.714059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b423e97418f474e99aa5b105bc6b3e6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:06.716061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:06.721342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:06.766814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b423e97418f474e99aa5b105bc6b3e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.779439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b423e97418f474e99aa5b105bc6b3e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.787734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b423e97418f474e99aa5b105bc6b3e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.788499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b423e97418f474e99aa5b105bc6b3e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.796092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e016f259-93a0-4320-b451-249700e461e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b423e97418f474e99aa5b105bc6b3e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:06.803236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 992/1978] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:06 2022] DELETE /identity/v3/users/5b423e97418f474e99aa5b105bc6b3e6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:07.158181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.159717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:07.175158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f379b11-1423-457b-b573-67b39ef5e3a6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.237711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.238413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.238696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.238971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] PATH_INFO: `/v3/projects/04c939821b324a4e883ffc38ac2f0a0b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.240124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c939821b324a4e883ffc38ac2f0a0b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.242030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:07.290348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c939821b324a4e883ffc38ac2f0a0b', 'request_id': 'req-0705b2b9-1012-4e30-b4c0-b136a8d9a143'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:07.291473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:07.297264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c939821b324a4e883ffc38ac2f0a0b', 'request_id': 'req-0705b2b9-1012-4e30-b4c0-b136a8d9a143'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:07.304303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c939821b324a4e883ffc38ac2f0a0b', 'request_id': 'req-0705b2b9-1012-4e30-b4c0-b136a8d9a143'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:07.316531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] The token cache is being invalidate because project 04c939821b324a4e883ffc38ac2f0a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:07.316820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:07.317441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0705b2b9-1012-4e30-b4c0-b136a8d9a143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:07.318921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 987/1979] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:07 2022] DELETE /identity/v3/projects/04c939821b324a4e883ffc38ac2f0a0b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:07.323750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe2cd4-c157-4cf0-aa5f-1425dcb83341 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.324228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe2cd4-c157-4cf0-aa5f-1425dcb83341 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.324350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe2cd4-c157-4cf0-aa5f-1425dcb83341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.345682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3abe2cd4-c157-4cf0-aa5f-1425dcb83341 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:07.346880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3abe2cd4-c157-4cf0-aa5f-1425dcb83341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:07.422568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 993/1980] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:07.424485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ca75-035f-49d8-8fbf-7888e6030ad8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.424852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ca75-035f-49d8-8fbf-7888e6030ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.425932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ca75-035f-49d8-8fbf-7888e6030ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.446968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3e15ca75-035f-49d8-8fbf-7888e6030ad8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:07.448403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3e15ca75-035f-49d8-8fbf-7888e6030ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:07.460876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 988/1981] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:07.461541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe2cd4-c157-4cf0-aa5f-1425dcb83341 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.488468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe2cd4-c157-4cf0-aa5f-1425dcb83341 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.495863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.496496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.496778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.497053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.497718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:07.498023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.499663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:07.515989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 994/1982] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:07.520817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ca75-035f-49d8-8fbf-7888e6030ad8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.531977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ca75-035f-49d8-8fbf-7888e6030ad8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.536882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.537554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.537901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.538239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.538950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.541239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:07.572063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 989/1983] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:07 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:07.576057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.587014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c18dbd-51cf-4a2c-964a-3fcc233ac1a5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.643971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.644605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.644889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.645167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.645804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.648026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:07.663766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:07.669620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 995/1984] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:07 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:07.673948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.684200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b89a95-b0a7-4bfa-8df3-fa44a20ea7bd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.689291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.689879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.690157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.690427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.691076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.692672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:07.702844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 990/1985] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:07.707342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.717263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ea11a-cc00-4add-9268-c217f01c6d7a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.722031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad21b1-6de3-43cb-8e36-4bed1eb42fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.722617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad21b1-6de3-43cb-8e36-4bed1eb42fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.722898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad21b1-6de3-43cb-8e36-4bed1eb42fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.723173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad21b1-6de3-43cb-8e36-4bed1eb42fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.723814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad21b1-6de3-43cb-8e36-4bed1eb42fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.725460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad21b1-6de3-43cb-8e36-4bed1eb42fc2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:07.736762 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 996/1986] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:07.740578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.750305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b7c27-aaa9-4ca0-9978-26465956460e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.755706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.756390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.756671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.756948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] PATH_INFO: `/v3/projects/05751e97a00e41df8e229e89950290d4/users/c1ba10cd6c43470a831ab2aa00b08146/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.772155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05751e97a00e41df8e229e89950290d4, user_id=c1ba10cd6c43470a831ab2aa00b08146, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.775056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:07.787595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 991/1987] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:07 2022] PUT /identity/v3/projects/05751e97a00e41df8e229e89950290d4/users/c1ba10cd6c43470a831ab2aa00b08146/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:07.798248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f827c-e04c-497f-8409-f445237d3c86 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.798565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f827c-e04c-497f-8409-f445237d3c86 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.798847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f827c-e04c-497f-8409-f445237d3c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.827232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-064f827c-e04c-497f-8409-f445237d3c86 None None] MFA Rules not processed for user `c1ba10cd6c43470a831ab2aa00b08146`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:07.828501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-064f827c-e04c-497f-8409-f445237d3c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:07.896753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 997/1988] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:07.901539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:07.963963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f56f8-ee27-4cf5-97a5-a18b6f948c2d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:07.968772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:07.969378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:07.969707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:07.969983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:07.970731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:07.972652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:08.000209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 992/1989] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:07 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:08.004775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f827c-e04c-497f-8409-f445237d3c86 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:08.015458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f827c-e04c-497f-8409-f445237d3c86 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:08.074810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:08.075342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:08.075621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:08.075927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:08.076553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:08.078817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:08.102994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 998/1990] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:08 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:08.106930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:08.117887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bce765-2fce-485c-94bc-598efa32d8c6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:08.122447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:08.123032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:08.123307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:08.123579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:08.124249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:08.125835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:08.136124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 993/1991] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:08.140142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:08.150321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97841-f6c6-4096-a244-aab6aae95b50 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:08.154932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:08.155572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:08.155860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:08.156148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] PATH_INFO: `/v3/projects/009534ac9beb4b0cb8fdba8181eaaf01/users/a809b7ae0e374ff1a4018547faf69d7d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:08.171716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009534ac9beb4b0cb8fdba8181eaaf01, user_id=a809b7ae0e374ff1a4018547faf69d7d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:08.175447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:08.188249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 999/1992] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:08 2022] PUT /identity/v3/projects/009534ac9beb4b0cb8fdba8181eaaf01/users/a809b7ae0e374ff1a4018547faf69d7d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:08.192345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:08.202154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33512a2c-ba3f-4b0b-a51d-2d126f23d77b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:08.260848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:08.274876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e96eb-023b-4f62-ba33-79e622ba7af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:08.275484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e96eb-023b-4f62-ba33-79e622ba7af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:08.275760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e96eb-023b-4f62-ba33-79e622ba7af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:08.276064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e96eb-023b-4f62-ba33-79e622ba7af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:08.276712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e96eb-023b-4f62-ba33-79e622ba7af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:08.278286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e96eb-023b-4f62-ba33-79e622ba7af6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:08.289110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 994/1993] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:08.293003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:08.304354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6e269-c7ac-4e4b-8ad1-cc7106e75527 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:08.309171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:08.309833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:08.310110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:08.310386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] PATH_INFO: `/v3/projects/009534ac9beb4b0cb8fdba8181eaaf01/users/a809b7ae0e374ff1a4018547faf69d7d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:08.323669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=009534ac9beb4b0cb8fdba8181eaaf01, user_id=a809b7ae0e374ff1a4018547faf69d7d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:08.323669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:08.325512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1000/1994] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:08 2022] PUT /identity/v3/projects/009534ac9beb4b0cb8fdba8181eaaf01/users/a809b7ae0e374ff1a4018547faf69d7d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:08.336708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65094e4b-290e-4665-976f-7739bfa31a14 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:08.340081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65094e4b-290e-4665-976f-7739bfa31a14 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:08.340640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65094e4b-290e-4665-976f-7739bfa31a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:08.371331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-65094e4b-290e-4665-976f-7739bfa31a14 None None] MFA Rules not processed for user `a809b7ae0e374ff1a4018547faf69d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:08.372736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-65094e4b-290e-4665-976f-7739bfa31a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:08.441570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 995/1995] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:08.452278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:08.453947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:08.488542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:08.513484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cb83d-bdfb-4f4d-b3f4-23867a688b6f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:08.517706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63833377-4c67-432a-ac4e-09e6814e72ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:08.518312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833377-4c67-432a-ac4e-09e6814e72ad service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:08.518600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833377-4c67-432a-ac4e-09e6814e72ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:08.518878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833377-4c67-432a-ac4e-09e6814e72ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:08.528259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63833377-4c67-432a-ac4e-09e6814e72ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:08.529666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63833377-4c67-432a-ac4e-09e6814e72ad service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:08.541435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1001/1996] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:09.169486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65094e4b-290e-4665-976f-7739bfa31a14 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:09.232926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65094e4b-290e-4665-976f-7739bfa31a14 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:09.237620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:09.238251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:09.238534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:09.238811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] PATH_INFO: `/v3/users/c1ba10cd6c43470a831ab2aa00b08146` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:09.239973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ba10cd6c43470a831ab2aa00b08146)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:09.241921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:09.293574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ba10cd6c43470a831ab2aa00b08146'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.305560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ba10cd6c43470a831ab2aa00b08146'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.306230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ba10cd6c43470a831ab2aa00b08146'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.313754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ba10cd6c43470a831ab2aa00b08146'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.314521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ba10cd6c43470a831ab2aa00b08146'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.316073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:09.328916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 996/1997] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:09 2022] DELETE /identity/v3/users/c1ba10cd6c43470a831ab2aa00b08146 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:09.334592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:09.334592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:09.335051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:09.355187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:09.356302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:09.429746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1002/1998] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:09.440730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:09.442109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:09.502413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90dc5-3c19-43d3-809e-eae1a0d1fad8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:09.506233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:09.506864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:09.507144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:09.507413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:09.517028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:09.518808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:09.528501 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 997/1999] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:09.834679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:09.835794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:09.849915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f024d-e2c1-46d8-af51-a4199f8518a3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:09.855150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:09.855776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:09.856130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:09.856407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] PATH_INFO: `/v3/projects/05751e97a00e41df8e229e89950290d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:09.857770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=05751e97a00e41df8e229e89950290d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:09.861394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:09.942607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05751e97a00e41df8e229e89950290d4', 'request_id': 'req-683c1557-758a-4d5c-b32b-2600bf5f59ac'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.949042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05751e97a00e41df8e229e89950290d4', 'request_id': 'req-683c1557-758a-4d5c-b32b-2600bf5f59ac'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.956621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05751e97a00e41df8e229e89950290d4', 'request_id': 'req-683c1557-758a-4d5c-b32b-2600bf5f59ac'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.974795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] The token cache is being invalidate because project 05751e97a00e41df8e229e89950290d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:09.975142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.975809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-683c1557-758a-4d5c-b32b-2600bf5f59ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:09.977569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1003/2000] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:09 2022] DELETE /identity/v3/projects/05751e97a00e41df8e229e89950290d4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:09.981699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:09.991935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219fb4f-81c8-41d9-9b6b-ea07932eed91 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:10.053685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:10.054415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.054682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.054953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] PATH_INFO: `/v3/users/a809b7ae0e374ff1a4018547faf69d7d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.056093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] RBAC: Authorizing `identity:delete_user(user_id=a809b7ae0e374ff1a4018547faf69d7d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:10.058022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:10.108050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a809b7ae0e374ff1a4018547faf69d7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.120161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a809b7ae0e374ff1a4018547faf69d7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.120829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a809b7ae0e374ff1a4018547faf69d7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.128375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a809b7ae0e374ff1a4018547faf69d7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.129023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-89c5ffe5-92aa-4f26-b992-172df003aade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a809b7ae0e374ff1a4018547faf69d7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.142459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 998/2001] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:09 2022] DELETE /identity/v3/users/a809b7ae0e374ff1a4018547faf69d7d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:10.146911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820f849-bf1f-44d5-989d-ef8cb200ab9d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.147201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820f849-bf1f-44d5-989d-ef8cb200ab9d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.147733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820f849-bf1f-44d5-989d-ef8cb200ab9d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.149264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1004/2002] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:05:10.158186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.158552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.158829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.197625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:10.198851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:10.205742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:10.289027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 999/2003] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:10.306499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820f849-bf1f-44d5-989d-ef8cb200ab9d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:10.309065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9820f849-bf1f-44d5-989d-ef8cb200ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:10.331713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820f849-bf1f-44d5-989d-ef8cb200ab9d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:10.337708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcccae-5276-471d-b695-fd3134be4754 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:10.338395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcccae-5276-471d-b695-fd3134be4754 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.338727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcccae-5276-471d-b695-fd3134be4754 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.338894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcccae-5276-471d-b695-fd3134be4754 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.349026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcccae-5276-471d-b695-fd3134be4754 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:10.350907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcccae-5276-471d-b695-fd3134be4754 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:10.365113 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1005/2004] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:10.550171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:10.551741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:10.634116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa59a9-10c2-4abc-97ba-0c258141ea5a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:10.638721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:10.639427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.639707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.640009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] PATH_INFO: `/v3/projects/009534ac9beb4b0cb8fdba8181eaaf01` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.641181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=009534ac9beb4b0cb8fdba8181eaaf01)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:10.644021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:10.724473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009534ac9beb4b0cb8fdba8181eaaf01', 'request_id': 'req-d9fb7e24-9660-49a9-868a-d905c898a3cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.731562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009534ac9beb4b0cb8fdba8181eaaf01', 'request_id': 'req-d9fb7e24-9660-49a9-868a-d905c898a3cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.747771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009534ac9beb4b0cb8fdba8181eaaf01', 'request_id': 'req-d9fb7e24-9660-49a9-868a-d905c898a3cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.761733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] The token cache is being invalidate because project 009534ac9beb4b0cb8fdba8181eaaf01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:10.762023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.762653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d9fb7e24-9660-49a9-868a-d905c898a3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:10.764191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1000/2005] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:10 2022] DELETE /identity/v3/projects/009534ac9beb4b0cb8fdba8181eaaf01 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:10.769872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5852f-d4a9-4f3b-98fd-d84aa3b7cc41 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.770203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5852f-d4a9-4f3b-98fd-d84aa3b7cc41 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.770445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5852f-d4a9-4f3b-98fd-d84aa3b7cc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.790783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c6d5852f-d4a9-4f3b-98fd-d84aa3b7cc41 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:10.793458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c6d5852f-d4a9-4f3b-98fd-d84aa3b7cc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:10.891900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:10.936421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1006/2006] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:10.937659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f78da7-15c7-4109-8f7b-f316c90d1463 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.938001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f78da7-15c7-4109-8f7b-f316c90d1463 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.938288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f78da7-15c7-4109-8f7b-f316c90d1463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.961682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f0f78da7-15c7-4109-8f7b-f316c90d1463 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:10.962920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f0f78da7-15c7-4109-8f7b-f316c90d1463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:10.974418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1001/2007] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:10.979210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5852f-d4a9-4f3b-98fd-d84aa3b7cc41 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:10.990092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5852f-d4a9-4f3b-98fd-d84aa3b7cc41 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:10.994703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:10.995316 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:10.995597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:10.995912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:10.996593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:10.996896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:10.998493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.012541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1007/2008] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:11.016968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f78da7-15c7-4109-8f7b-f316c90d1463 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.027506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f78da7-15c7-4109-8f7b-f316c90d1463 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:11.032782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:11.033437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.033727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.034125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.035800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:11.036794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.068284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1002/2009] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:11 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:11.073008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.084771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef870c-3cc6-4f79-83d9-610727612bba admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:11.153077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:11.154011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.154011 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.154214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.154786 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:11.156775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.180869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1008/2010] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:11 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:11.185078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.192189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:11.196554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c31546-c3e3-4dd2-a5ee-b6192ade025e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:11.205168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:11.205780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.206059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.206332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.207011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:11.208744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.221521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1003/2011] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:11.225449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.235973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5ab07-5bbb-4ea2-94ee-4550cfd88689 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:11.240636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3a16f-1885-4081-a8a3-340086034712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:11.241237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3a16f-1885-4081-a8a3-340086034712 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.241518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3a16f-1885-4081-a8a3-340086034712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.241793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3a16f-1885-4081-a8a3-340086034712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.242436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3a16f-1885-4081-a8a3-340086034712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:11.244112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3a16f-1885-4081-a8a3-340086034712 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.255243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1009/2012] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:11.259955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.270049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140e0a7-f0ab-4252-991a-71e5ab8c0304 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:11.275254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:11.275939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.276225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.276494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] PATH_INFO: `/v3/projects/e9089469233842f1ac3a52cd2c53f108/users/f69f2ffda8414d41965b42b1b060385f/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.292305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9089469233842f1ac3a52cd2c53f108, user_id=f69f2ffda8414d41965b42b1b060385f, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:11.295012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.309048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1004/2013] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:11 2022] PUT /identity/v3/projects/e9089469233842f1ac3a52cd2c53f108/users/f69f2ffda8414d41965b42b1b060385f/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:11.320577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772b25c-98c6-40d4-a8e6-66d1453a0217 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.320865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772b25c-98c6-40d4-a8e6-66d1453a0217 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.321144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772b25c-98c6-40d4-a8e6-66d1453a0217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.349689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4772b25c-98c6-40d4-a8e6-66d1453a0217 None None] MFA Rules not processed for user `f69f2ffda8414d41965b42b1b060385f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:11.350874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4772b25c-98c6-40d4-a8e6-66d1453a0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:11.424731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1010/2014] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:11.435846 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.437536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:11.499870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a48f05-312d-445b-83f2-1d680bea6b13 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:11.504289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1d3a1-36f9-40a0-85cb-c9e828cdc3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:11.504904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d3a1-36f9-40a0-85cb-c9e828cdc3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.505182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d3a1-36f9-40a0-85cb-c9e828cdc3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.505450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1d3a1-36f9-40a0-85cb-c9e828cdc3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.521077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d3a1-36f9-40a0-85cb-c9e828cdc3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:11.522807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1d3a1-36f9-40a0-85cb-c9e828cdc3c3 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.538673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1005/2015] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:11.560522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772b25c-98c6-40d4-a8e6-66d1453a0217 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.633305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:11.671819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772b25c-98c6-40d4-a8e6-66d1453a0217 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:11.679241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:11.679959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.680231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.680508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] PATH_INFO: `/v3/users/f69f2ffda8414d41965b42b1b060385f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.681899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69f2ffda8414d41965b42b1b060385f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:11.683994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:11.749671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69f2ffda8414d41965b42b1b060385f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:11.765060 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69f2ffda8414d41965b42b1b060385f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:11.775882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69f2ffda8414d41965b42b1b060385f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:11.776909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69f2ffda8414d41965b42b1b060385f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:11.788613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69f2ffda8414d41965b42b1b060385f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:11.797055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1011/2016] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:11 2022] DELETE /identity/v3/users/f69f2ffda8414d41965b42b1b060385f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:11.805262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:11.805677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:11.806225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:11.828491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:11.830397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:11.955898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1006/2017] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:11.975064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:11.977317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:12.051367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af370088-c9df-4de0-9539-65cc49506e71 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:12.055501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437e20-7c5c-45bb-86cc-fa8272c56fa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:12.056142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437e20-7c5c-45bb-86cc-fa8272c56fa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.056471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437e20-7c5c-45bb-86cc-fa8272c56fa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.056747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437e20-7c5c-45bb-86cc-fa8272c56fa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.071257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437e20-7c5c-45bb-86cc-fa8272c56fa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:12.073036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437e20-7c5c-45bb-86cc-fa8272c56fa1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:12.085727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1012/2018] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:12.508487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:12.509962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:12.522938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75e4e8-d997-4888-acb1-e44f7527b8f1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:12.527791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:12.528484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.528764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.529040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] PATH_INFO: `/v3/projects/e9089469233842f1ac3a52cd2c53f108` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.530164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9089469233842f1ac3a52cd2c53f108)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:12.532759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:12.579683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9089469233842f1ac3a52cd2c53f108', 'request_id': 'req-0249cf42-1864-40b2-98e0-833bfbba7471'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:12.585281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9089469233842f1ac3a52cd2c53f108', 'request_id': 'req-0249cf42-1864-40b2-98e0-833bfbba7471'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:12.591502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9089469233842f1ac3a52cd2c53f108', 'request_id': 'req-0249cf42-1864-40b2-98e0-833bfbba7471'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:12.595897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:12.607304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] The token cache is being invalidate because project e9089469233842f1ac3a52cd2c53f108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:12.607591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:12.608234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0249cf42-1864-40b2-98e0-833bfbba7471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:12.609740 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1007/2019] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:12 2022] DELETE /identity/v3/projects/e9089469233842f1ac3a52cd2c53f108 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:12.614922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760826c-8e1d-4ac1-9f9f-6d6d62dea2ae None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.615439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760826c-8e1d-4ac1-9f9f-6d6d62dea2ae None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.615536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760826c-8e1d-4ac1-9f9f-6d6d62dea2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.635128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f760826c-8e1d-4ac1-9f9f-6d6d62dea2ae None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:12.636331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f760826c-8e1d-4ac1-9f9f-6d6d62dea2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:12.697072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1013/2020] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:12.705437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f686a3e-7be0-4122-b21d-f261e5af1bf1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.705654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f686a3e-7be0-4122-b21d-f261e5af1bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.705935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f686a3e-7be0-4122-b21d-f261e5af1bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.726660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5f686a3e-7be0-4122-b21d-f261e5af1bf1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:12.727893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5f686a3e-7be0-4122-b21d-f261e5af1bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:12.736129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1008/2021] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:12.740565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760826c-8e1d-4ac1-9f9f-6d6d62dea2ae None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:12.745517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:12.753840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760826c-8e1d-4ac1-9f9f-6d6d62dea2ae None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:12.760186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:12.760805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.761081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.761356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.762061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:12.762371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:12.763996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:12.780218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1014/2022] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:12.784694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f686a3e-7be0-4122-b21d-f261e5af1bf1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:12.793926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f686a3e-7be0-4122-b21d-f261e5af1bf1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:12.798717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:12.799308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.799589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.799883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.800849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:12.802449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:12.833425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1009/2023] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:12 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:12.834611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:12.844599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942f68d-d573-4db8-a2d4-7eef450db830 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:12.900646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:12.901251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.901527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.901796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.902436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:12.904372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:12.924945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1015/2024] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:12 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:12.929658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:12.940431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6728-3932-45f7-9c8d-b2ffbc257430 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:12.945383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:12.945993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.946276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.946552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.947226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:12.948868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:12.960971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1010/2025] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:12.965137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:12.976637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908bd0-9157-4a58-8116-7cb412036054 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:12.984869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112cc9b-95b5-4043-ba14-1026a583bd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:12.986007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112cc9b-95b5-4043-ba14-1026a583bd16 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:12.986319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112cc9b-95b5-4043-ba14-1026a583bd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:12.986541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112cc9b-95b5-4043-ba14-1026a583bd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:12.987375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112cc9b-95b5-4043-ba14-1026a583bd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:12.989035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112cc9b-95b5-4043-ba14-1026a583bd16 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:13.001544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1016/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:13.005798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:13.016407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca538a-d6bd-4b6d-a772-fcf92d4a46e3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:13.031545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:13.031545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:13.031545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:13.031545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] PATH_INFO: `/v3/projects/6079df79fc1b485cb130e81f61dac695/users/6029b853df904de6ad16e209dd237a29/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:13.042507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6079df79fc1b485cb130e81f61dac695, user_id=6029b853df904de6ad16e209dd237a29, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:13.045227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:13.061905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1011/2027] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:13 2022] PUT /identity/v3/projects/6079df79fc1b485cb130e81f61dac695/users/6029b853df904de6ad16e209dd237a29/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:13.075938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:13.076507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:13.079884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:13.108159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] MFA Rules not processed for user `6029b853df904de6ad16e209dd237a29`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:13.109435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:13.179887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1017/2028] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:13.191551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:13.193277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:13.258581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fbb0-e67d-431c-98c5-d95912c8aa54 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:13.267043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de86a8-1ae1-45e2-94b5-6cccccdd99e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:13.267776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de86a8-1ae1-45e2-94b5-6cccccdd99e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:13.268222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de86a8-1ae1-45e2-94b5-6cccccdd99e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:13.268639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de86a8-1ae1-45e2-94b5-6cccccdd99e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:13.277851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de86a8-1ae1-45e2-94b5-6cccccdd99e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:13.279270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de86a8-1ae1-45e2-94b5-6cccccdd99e1 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:13.287941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1012/2029] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:18.816348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:18.817859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:18.917925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:18.925819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbb896-b57b-4546-8f48-9319db4db6f0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:18.934013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:18.934797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:18.934936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:18.935212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] PATH_INFO: `/v3/users/6c2e5626258a4f62a3004680eb93904f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:18.936380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2e5626258a4f62a3004680eb93904f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:18.939400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:19.016372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2e5626258a4f62a3004680eb93904f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.038660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2e5626258a4f62a3004680eb93904f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.052787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2e5626258a4f62a3004680eb93904f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.053518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2e5626258a4f62a3004680eb93904f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.073452 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2e5626258a4f62a3004680eb93904f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.084172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1018/2030] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:18 2022] DELETE /identity/v3/users/6c2e5626258a4f62a3004680eb93904f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:19.089196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:19.090037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:19.090605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:19.128384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:19.129797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:19.205149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1013/2031] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:19.216865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:19.218595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:19.323301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932abd5-3bfa-4019-88a9-904e3416c01c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:19.327679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:19.328343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:19.328612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:19.329590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:19.343822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:19.346367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:19.356586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1019/2032] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:19.745473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:19.749235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:19.761863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c966c-477e-4c60-a92a-fc153d4b79c6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:19.766657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:19.767444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:19.767731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:19.768065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] PATH_INFO: `/v3/projects/fc6e235982a24981b5997b8bf42d59b2` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:19.769202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc6e235982a24981b5997b8bf42d59b2)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:19.771193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:19.805667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:19.822362 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc6e235982a24981b5997b8bf42d59b2', 'request_id': 'req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.828663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc6e235982a24981b5997b8bf42d59b2', 'request_id': 'req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.835875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc6e235982a24981b5997b8bf42d59b2', 'request_id': 'req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.849014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] The token cache is being invalidate because project fc6e235982a24981b5997b8bf42d59b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:19.849310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.849920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.851411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1014/2033] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:19 2022] DELETE /identity/v3/projects/fc6e235982a24981b5997b8bf42d59b2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:19.855421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:19.866086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766dab46-e5dc-4922-9272-a1d876af19d0 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:19.925495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:19.926136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:19.926415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:19.926691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] PATH_INFO: `/v3/users/a9895ba13a4e4c01a6fee986f29cb74b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:19.927812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9895ba13a4e4c01a6fee986f29cb74b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:19.929775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:19.952536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:19.978412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9895ba13a4e4c01a6fee986f29cb74b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.990682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9895ba13a4e4c01a6fee986f29cb74b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.997863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9895ba13a4e4c01a6fee986f29cb74b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:19.998563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9895ba13a4e4c01a6fee986f29cb74b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:20.006886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8f4d9d2f-54c1-49a8-a819-9aee52dcc8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9895ba13a4e4c01a6fee986f29cb74b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:20.012741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1020/2034] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:19 2022] DELETE /identity/v3/users/a9895ba13a4e4c01a6fee986f29cb74b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:20.301236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:20.302743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:20.317330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8fe9b-74f0-4f7b-afb0-49a3185b7d42 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:20.394066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:20.394719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.395098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.395381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] PATH_INFO: `/v3/projects/8b2afc78752d4170b555e0f1a3b43786` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.408404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2afc78752d4170b555e0f1a3b43786)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:20.408404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:20.464395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2afc78752d4170b555e0f1a3b43786', 'request_id': 'req-e67263e8-6933-4b69-89e6-1dd155502ddb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:20.470259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2afc78752d4170b555e0f1a3b43786', 'request_id': 'req-e67263e8-6933-4b69-89e6-1dd155502ddb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:20.477255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2afc78752d4170b555e0f1a3b43786', 'request_id': 'req-e67263e8-6933-4b69-89e6-1dd155502ddb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:20.493269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] The token cache is being invalidate because project 8b2afc78752d4170b555e0f1a3b43786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:20.493562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:20.494218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e67263e8-6933-4b69-89e6-1dd155502ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:20.496441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1015/2035] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:20 2022] DELETE /identity/v3/projects/8b2afc78752d4170b555e0f1a3b43786 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:20.502297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a082cc-4272-45a8-814f-7b3d678dc0bc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.502568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a082cc-4272-45a8-814f-7b3d678dc0bc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.502844 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a082cc-4272-45a8-814f-7b3d678dc0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.524977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46a082cc-4272-45a8-814f-7b3d678dc0bc None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:20.526190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46a082cc-4272-45a8-814f-7b3d678dc0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:20.590222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1021/2036] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:20.601151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8109d6-6226-405a-871a-8733b1379e2d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.601438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8109d6-6226-405a-871a-8733b1379e2d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.601749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8109d6-6226-405a-871a-8733b1379e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.623561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4d8109d6-6226-405a-871a-8733b1379e2d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:20.624886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4d8109d6-6226-405a-871a-8733b1379e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:20.636289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1016/2037] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:20.640514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a082cc-4272-45a8-814f-7b3d678dc0bc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:20.652023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a082cc-4272-45a8-814f-7b3d678dc0bc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:20.657051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:20.657628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.657960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.658240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.659210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:20.659790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:20.662111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:20.682069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1022/2038] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:20.683260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8109d6-6226-405a-871a-8733b1379e2d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:20.694038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8109d6-6226-405a-871a-8733b1379e2d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:20.698802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:20.699417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.699695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.700008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.700654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:20.702723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:20.735696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1017/2039] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:20 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:20.739931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:20.752427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c83923-159a-46fa-b89e-4724db328e4f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:20.824258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:20.824885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.825166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.825440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.826080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:20.828237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:20.857674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1023/2040] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:20 2022] POST /identity/v3/users => generated 305 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:20.862122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:20.875447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7203747-adc4-417e-91e1-5a66fc6d4571 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:20.881794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:20.882726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.883459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.884029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.885234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:20.888157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:20.902373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1018/2041] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:20.906512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:20.917646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bad24-266b-4070-a2cc-462bee6b3ebb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:20.924029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137f98d-349b-4555-b86c-4cdd6805f2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:20.924659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137f98d-349b-4555-b86c-4cdd6805f2da admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.925040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137f98d-349b-4555-b86c-4cdd6805f2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.925323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137f98d-349b-4555-b86c-4cdd6805f2da admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.926116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137f98d-349b-4555-b86c-4cdd6805f2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:20.927724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137f98d-349b-4555-b86c-4cdd6805f2da admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:20.939698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1024/2042] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:20.944695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:20.956125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c79c7-3aa5-467c-972a-57831f00f87e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:20.960832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:20.961521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:20.961800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:20.962224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] PATH_INFO: `/v3/projects/9c88c47136644d29a80116a1f94fa1c0/users/f9355c07ee494faeb99d32dc3d48cbf5/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:20.968247 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:20.983453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c88c47136644d29a80116a1f94fa1c0, user_id=f9355c07ee494faeb99d32dc3d48cbf5, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:20.987955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.001755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1019/2043] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:20 2022] PUT /identity/v3/projects/9c88c47136644d29a80116a1f94fa1c0/users/f9355c07ee494faeb99d32dc3d48cbf5/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:21.012173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0505-d649-4ebb-948d-5eabee6eb53c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.012463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0505-d649-4ebb-948d-5eabee6eb53c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.012740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0505-d649-4ebb-948d-5eabee6eb53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.021939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:21.045260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dd4b0505-d649-4ebb-948d-5eabee6eb53c None None] MFA Rules not processed for user `f9355c07ee494faeb99d32dc3d48cbf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:21.046481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd4b0505-d649-4ebb-948d-5eabee6eb53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:21.133883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1025/2044] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:21.138687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.208062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bffff-aa15-40eb-936f-d70219d79d5c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.212739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.213340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.213618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.213889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.214521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:21.216446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.252024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1020/2045] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:21 2022] POST /identity/v3/projects => generated 407 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:21.256172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b0505-d649-4ebb-948d-5eabee6eb53c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.267501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b0505-d649-4ebb-948d-5eabee6eb53c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.330018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.330623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.330900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.331171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.331798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:21.333723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.355831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1026/2046] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:21 2022] POST /identity/v3/users => generated 310 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:21.359983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.370744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f4b98-f993-4209-90d3-e9e0fd4b4e54 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.375274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.375894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.376181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.376451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.377116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:21.378714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.389329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1021/2047] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:21.394324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.404944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932292f-cf5c-4d82-8e63-6d0452a5836b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.410212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.410870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.411152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.411430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] PATH_INFO: `/v3/projects/7800c9caeb0a4fe186a0e9d27a56300c/users/548057e4dfd348899ff1dfc481c4f1a3/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.428178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7800c9caeb0a4fe186a0e9d27a56300c, user_id=548057e4dfd348899ff1dfc481c4f1a3, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:21.430880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.445475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1027/2048] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:21 2022] PUT /identity/v3/projects/7800c9caeb0a4fe186a0e9d27a56300c/users/548057e4dfd348899ff1dfc481c4f1a3/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:21.449510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.460202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b462fa-fdb1-4cec-87ff-ce020f16cc59 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.515734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710380c-f00e-45d6-8c0d-4038ad766588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.516880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710380c-f00e-45d6-8c0d-4038ad766588 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.517391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710380c-f00e-45d6-8c0d-4038ad766588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.517713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710380c-f00e-45d6-8c0d-4038ad766588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.518352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710380c-f00e-45d6-8c0d-4038ad766588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:21.520032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710380c-f00e-45d6-8c0d-4038ad766588 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.530208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1022/2049] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:21.534922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.545188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f83e-0486-49cb-9abd-be248242769f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.549949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.550602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.550878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.551148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] PATH_INFO: `/v3/projects/7800c9caeb0a4fe186a0e9d27a56300c/users/548057e4dfd348899ff1dfc481c4f1a3/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.553053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7800c9caeb0a4fe186a0e9d27a56300c, user_id=548057e4dfd348899ff1dfc481c4f1a3, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:21.555678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.566875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1028/2050] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:21 2022] PUT /identity/v3/projects/7800c9caeb0a4fe186a0e9d27a56300c/users/548057e4dfd348899ff1dfc481c4f1a3/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:21.577125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddf53b-dfd8-410e-8697-16887431073f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.577423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddf53b-dfd8-410e-8697-16887431073f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.577705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddf53b-dfd8-410e-8697-16887431073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.608068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5dddf53b-dfd8-410e-8697-16887431073f None None] MFA Rules not processed for user `548057e4dfd348899ff1dfc481c4f1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:21.609681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5dddf53b-dfd8-410e-8697-16887431073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:21.654305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:21.681202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1023/2051] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:21.692166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.695554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:21.748829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:21.755685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a85c6e-464d-4918-bfed-a4e8cb75f09b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.759448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.760088 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.760372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.760643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:21.770365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:21.771925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:21.781385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1029/2052] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:21 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:21.961782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddf53b-dfd8-410e-8697-16887431073f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:21.971327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddf53b-dfd8-410e-8697-16887431073f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:21.991963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d0d58-ff21-43b7-98cd-23ccb9efd858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:21.993307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d0d58-ff21-43b7-98cd-23ccb9efd858 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:21.993735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d0d58-ff21-43b7-98cd-23ccb9efd858 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:21.994074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d0d58-ff21-43b7-98cd-23ccb9efd858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:22.069520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d0d58-ff21-43b7-98cd-23ccb9efd858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:22.071301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d0d58-ff21-43b7-98cd-23ccb9efd858 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:22.081063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1024/2053] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:21 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:31.777251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:32.076405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=37 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:34.862709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:34.864437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:34.948777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83524d52-cfdd-4405-abd3-b2660b731763 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:34.957372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:34.958031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:34.958312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:34.958619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] PATH_INFO: `/v3/users/c0734b9ac75648609f7a4ce17a48d9ce` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:34.959938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0734b9ac75648609f7a4ce17a48d9ce)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:34.961875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:35.015721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0734b9ac75648609f7a4ce17a48d9ce'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:35.030985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0734b9ac75648609f7a4ce17a48d9ce'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:35.040347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0734b9ac75648609f7a4ce17a48d9ce'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:35.041006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0734b9ac75648609f7a4ce17a48d9ce'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:35.048399 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0734b9ac75648609f7a4ce17a48d9ce'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:35.055797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1030/2054] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:34 2022] DELETE /identity/v3/users/c0734b9ac75648609f7a4ce17a48d9ce => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:35.060192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:35.060710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:35.060866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:35.083030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:35.084388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:35.171143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1025/2055] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:35.181072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:35.182524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:35.249442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d54e70-1b5e-4ced-839e-a5a815e871c1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:35.254385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:35.255075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:35.255350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:35.255624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:35.265882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:35.268181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:35.278260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1031/2056] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:35.961312 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:35.962865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:35.976904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c98d49-2a39-4f10-889a-f68751b539db None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:35.981628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:35.982771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:35.982771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:35.982953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] PATH_INFO: `/v3/projects/6de799cf25d04709a5d88e3d755aad6a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:35.984127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de799cf25d04709a5d88e3d755aad6a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:35.986713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:36.046313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de799cf25d04709a5d88e3d755aad6a', 'request_id': 'req-f8e1058c-fa03-48f2-b30c-5f4bce232c27'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.054107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de799cf25d04709a5d88e3d755aad6a', 'request_id': 'req-f8e1058c-fa03-48f2-b30c-5f4bce232c27'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.061436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de799cf25d04709a5d88e3d755aad6a', 'request_id': 'req-f8e1058c-fa03-48f2-b30c-5f4bce232c27'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.078852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] The token cache is being invalidate because project 6de799cf25d04709a5d88e3d755aad6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:36.079851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.080573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.085751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1026/2057] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:35 2022] DELETE /identity/v3/projects/6de799cf25d04709a5d88e3d755aad6a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:36.086836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:36.098653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7c72d-2f29-4bac-ab04-754857cdc2f6 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:36.122068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:36.189978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:36.190916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:36.191207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:36.191482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] PATH_INFO: `/v3/users/3fb1e7ba68574d95956dfa2ec3741b44` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:36.192658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb1e7ba68574d95956dfa2ec3741b44)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:36.194579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:36.259976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb1e7ba68574d95956dfa2ec3741b44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.276331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb1e7ba68574d95956dfa2ec3741b44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.293389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb1e7ba68574d95956dfa2ec3741b44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.294305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb1e7ba68574d95956dfa2ec3741b44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.305602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6beadda-4bd5-4fdb-bf2b-47cfeee825e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb1e7ba68574d95956dfa2ec3741b44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.313314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1032/2058] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:36 2022] DELETE /identity/v3/users/3fb1e7ba68574d95956dfa2ec3741b44 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:36.656580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:36.658171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:36.675498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1058c-fa03-48f2-b30c-5f4bce232c27 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:36.814916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:36.815587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:36.815899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:36.816218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] PATH_INFO: `/v3/projects/501fdf19d46b4f92b9ae34d01fa2fddd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:36.817483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=501fdf19d46b4f92b9ae34d01fa2fddd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:36.820187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:36.862542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:36.901258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '501fdf19d46b4f92b9ae34d01fa2fddd', 'request_id': 'req-798da0f9-128b-4588-8b6e-1350f49f20d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.909832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '501fdf19d46b4f92b9ae34d01fa2fddd', 'request_id': 'req-798da0f9-128b-4588-8b6e-1350f49f20d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.919026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '501fdf19d46b4f92b9ae34d01fa2fddd', 'request_id': 'req-798da0f9-128b-4588-8b6e-1350f49f20d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.926913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60171f02-41bd-4fd5-ab24-b959b437dbb8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:36.927231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60171f02-41bd-4fd5-ab24-b959b437dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:36.927533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60171f02-41bd-4fd5-ab24-b959b437dbb8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:36.929329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1033/2059] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:05:36.939992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] The token cache is being invalidate because project 501fdf19d46b4f92b9ae34d01fa2fddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:36.940291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.940974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-798da0f9-128b-4588-8b6e-1350f49f20d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:36.943885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1027/2060] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:36 2022] DELETE /identity/v3/projects/501fdf19d46b4f92b9ae34d01fa2fddd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:36.947973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c0279-8e5c-42f7-a431-93472c120b79 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:36.948265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c0279-8e5c-42f7-a431-93472c120b79 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:36.948958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c0279-8e5c-42f7-a431-93472c120b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:36.979248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-744c0279-8e5c-42f7-a431-93472c120b79 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:36.980783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-744c0279-8e5c-42f7-a431-93472c120b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:36.999751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10b8f1-3f8e-4576-9ebb-9d01cfca7a95 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.000099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10b8f1-3f8e-4576-9ebb-9d01cfca7a95 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.000957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10b8f1-3f8e-4576-9ebb-9d01cfca7a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.030539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5d10b8f1-3f8e-4576-9ebb-9d01cfca7a95 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:37.032116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5d10b8f1-3f8e-4576-9ebb-9d01cfca7a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:37.129687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1034/2061] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:37.131748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec5b5-33b0-47ec-9b9f-9b5c3d84a52b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.132276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec5b5-33b0-47ec-9b9f-9b5c3d84a52b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.132984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec5b5-33b0-47ec-9b9f-9b5c3d84a52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.160579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1028/2062] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:05:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:37.162143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e0eec5b5-33b0-47ec-9b9f-9b5c3d84a52b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:37.163893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e0eec5b5-33b0-47ec-9b9f-9b5c3d84a52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:37.178244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51b09-20e7-4c4b-82c0-d934727a5bf2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.178700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51b09-20e7-4c4b-82c0-d934727a5bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.179113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51b09-20e7-4c4b-82c0-d934727a5bf2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.180402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1035/2063] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:37.180814 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1029/2064] 10.222.0.52 () {54 vars in 961 bytes} [Thu Oct 6 11:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 11:05:37.183540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eec5b5-33b0-47ec-9b9f-9b5c3d84a52b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.186758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa135cfc-1403-4d79-8f46-3155ccdbf903 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.187260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa135cfc-1403-4d79-8f46-3155ccdbf903 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.188795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa135cfc-1403-4d79-8f46-3155ccdbf903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.195948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eec5b5-33b0-47ec-9b9f-9b5c3d84a52b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.202782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.203659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.216072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-aa135cfc-1403-4d79-8f46-3155ccdbf903 None None] MFA Rules not processed for user `49cbfb2f4ddb42c0b299311615c7b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:37.216221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.217520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.218754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aa135cfc-1403-4d79-8f46-3155ccdbf903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:37.220376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:37.220837 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.224061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.246210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1030/2065] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Oct 6 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:37.252659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1036/2066] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:37.253415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa135cfc-1403-4d79-8f46-3155ccdbf903 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.258947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.266593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa135cfc-1403-4d79-8f46-3155ccdbf903 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.272826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.273638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.274149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.274685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.275526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.276428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba025ea-f8e7-4e0f-97da-2c42d1cf0c05 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.277930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.285130 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.285805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.286033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.286307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.297100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.299883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.311875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1037/2067] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:37.315491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1031/2068] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:37 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:37.321077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.333995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090907e-7bee-4d77-a3c7-f419d82e1605 service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.341166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:37.418251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.418884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.419162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.419506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.420174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.422889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.444446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1038/2069] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:37 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:37.448628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.459454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6604-2637-4a97-b7c2-1c5ceedf6aba admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.464432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.465131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.465825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.465825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.466753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.471274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.483873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1032/2070] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:37.491563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.503290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854b780-0965-473a-bf56-cd8213fb749e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.510526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bef69-b41f-4648-8531-dbb271d911de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.511678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bef69-b41f-4648-8531-dbb271d911de admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.512282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bef69-b41f-4648-8531-dbb271d911de admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.512874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bef69-b41f-4648-8531-dbb271d911de admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.513924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bef69-b41f-4648-8531-dbb271d911de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.515853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bef69-b41f-4648-8531-dbb271d911de admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.529167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1039/2071] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:37.534119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.549456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db7838-a276-4d2c-ac29-41afbd2062f4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.554735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.555392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.555668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.555966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] PATH_INFO: `/v3/projects/ab8fa422354844b1b9291d2b2d6d2ea5/users/fe7ab77daeb242248c6d6bf80ce6779c/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.571904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab8fa422354844b1b9291d2b2d6d2ea5, user_id=fe7ab77daeb242248c6d6bf80ce6779c, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.574570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.591617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1033/2072] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:37 2022] PUT /identity/v3/projects/ab8fa422354844b1b9291d2b2d6d2ea5/users/fe7ab77daeb242248c6d6bf80ce6779c/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:37.603890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb2b1-5ab9-49bc-becb-012ebf1782f5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.604237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb2b1-5ab9-49bc-becb-012ebf1782f5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.606931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb2b1-5ab9-49bc-becb-012ebf1782f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.641649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8bdeb2b1-5ab9-49bc-becb-012ebf1782f5 None None] MFA Rules not processed for user `fe7ab77daeb242248c6d6bf80ce6779c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:37.643393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8bdeb2b1-5ab9-49bc-becb-012ebf1782f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:37.745293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1040/2073] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:37.749883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.822980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:37.867734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd6a1a-9377-42c6-b42a-ce158df65e8c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.872666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:37.873301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:37.873572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:37.873848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:37.874699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:37.877160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:37.918639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1034/2074] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:37 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:37.920348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeb2b1-5ab9-49bc-becb-012ebf1782f5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:37.957275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeb2b1-5ab9-49bc-becb-012ebf1782f5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:37.998235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.000295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.143886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.144642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.145049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.145425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.146206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.147885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737aea2-e930-4d86-a16e-e5c0c4702904 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.148734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.159124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.160527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.161009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.161579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] PATH_INFO: `/v3/users/f5d3b9aad894446ba60090dde495571c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.162773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d3b9aad894446ba60090dde495571c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.169015 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.177564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1041/2075] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:37 2022] POST /identity/v3/users => generated 299 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:38.181300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.202534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b97970-b481-45af-abbc-6db7e93753b3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.211035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.211772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.212323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.213234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.214304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.215934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.217899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:38.231387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1042/2076] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:38.232785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.235399 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.238933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d3b9aad894446ba60090dde495571c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.256209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d3b9aad894446ba60090dde495571c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.257211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d3b9aad894446ba60090dde495571c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.260332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8019f-7cd3-44bb-be8b-4b4c00663016 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.267360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.268144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.268344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d3b9aad894446ba60090dde495571c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.268591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.269188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] PATH_INFO: `/v3/users/f9355c07ee494faeb99d32dc3d48cbf5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.270424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9355c07ee494faeb99d32dc3d48cbf5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.271177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d3b9aad894446ba60090dde495571c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.272569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.293439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1035/2077] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:37 2022] DELETE /identity/v3/users/f5d3b9aad894446ba60090dde495571c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:38.294439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.296621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.362394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9355c07ee494faeb99d32dc3d48cbf5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.376581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:38.377201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9355c07ee494faeb99d32dc3d48cbf5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.395564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9355c07ee494faeb99d32dc3d48cbf5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.396319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9355c07ee494faeb99d32dc3d48cbf5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.400777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec066a-12c0-4fc3-b478-c5fb31c7527d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.405166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6a3ca65-8bd3-498a-b018-76b97d999b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9355c07ee494faeb99d32dc3d48cbf5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.410401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.411112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.413078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.414792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1043/2078] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:38 2022] DELETE /identity/v3/users/f9355c07ee494faeb99d32dc3d48cbf5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:38.414923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] PATH_INFO: `/v3/projects/6e35e344ed0c49b79a7fc3c2f946ab5f/users/887f874cd79442b89313b05b3f80f4d5/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.418951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.419455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.419568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.440285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e35e344ed0c49b79a7fc3c2f946ab5f, user_id=887f874cd79442b89313b05b3f80f4d5, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.441440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:38.442961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.444210 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.466126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1036/2079] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:38 2022] PUT /identity/v3/projects/6e35e344ed0c49b79a7fc3c2f946ab5f/users/887f874cd79442b89313b05b3f80f4d5/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:38.470445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.473130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.524645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1044/2080] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:38.547992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad6ae-7839-4597-9c4c-d09a95bb98d6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.553273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.553989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.554368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.554703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.555425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.557160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.569993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1037/2081] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:38.570640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.572949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.584352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.603264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35383d44-2217-4b8c-9c57-51f22d3e0a51 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.618755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.619832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.620154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.620494 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] PATH_INFO: `/v3/projects/6e35e344ed0c49b79a7fc3c2f946ab5f/users/887f874cd79442b89313b05b3f80f4d5/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.622771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e35e344ed0c49b79a7fc3c2f946ab5f, user_id=887f874cd79442b89313b05b3f80f4d5, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.625678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.639874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1038/2082] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:38 2022] PUT /identity/v3/projects/6e35e344ed0c49b79a7fc3c2f946ab5f/users/887f874cd79442b89313b05b3f80f4d5/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:38.643975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.645982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.712253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14530f39-c3aa-4f8e-a713-7d7d6b8ac845 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.727466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059c726-5b56-42f9-ac92-c3ad34d0b502 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.734134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.734784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:38.735074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.735416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.735750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] PATH_INFO: `/v3/projects/988b46704eed4dea920dbd0db5c1dd0b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.736975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] RBAC: Authorizing `identity:delete_project(project_id=988b46704eed4dea920dbd0db5c1dd0b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.739427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.771915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ee74a-363b-4113-aad9-fc9584bdb2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.773013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ee74a-363b-4113-aad9-fc9584bdb2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.773395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ee74a-363b-4113-aad9-fc9584bdb2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.773838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ee74a-363b-4113-aad9-fc9584bdb2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.787328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ee74a-363b-4113-aad9-fc9584bdb2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.789055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ee74a-363b-4113-aad9-fc9584bdb2b1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.795670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988b46704eed4dea920dbd0db5c1dd0b', 'request_id': 'req-bd790a0c-67a0-4239-8e63-b195beab317f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.800153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1045/2083] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:38.801432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.802055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.802173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988b46704eed4dea920dbd0db5c1dd0b', 'request_id': 'req-bd790a0c-67a0-4239-8e63-b195beab317f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.802293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.812085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988b46704eed4dea920dbd0db5c1dd0b', 'request_id': 'req-bd790a0c-67a0-4239-8e63-b195beab317f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.814355 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:38.832605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] The token cache is being invalidate because project 988b46704eed4dea920dbd0db5c1dd0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:38.832903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.833584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:38.836193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1039/2084] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:38 2022] DELETE /identity/v3/projects/988b46704eed4dea920dbd0db5c1dd0b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:38.839996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.841474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] MFA Rules not processed for user `887f874cd79442b89313b05b3f80f4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:38.841832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.843486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:38.928760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd790a0c-67a0-4239-8e63-b195beab317f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:38.936514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:38.937316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:38.938576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:38.938991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] PATH_INFO: `/v3/users/f51678a19f8f4f7bb62990809729d8b4` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:38.940678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51678a19f8f4f7bb62990809729d8b4)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:38.943197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:38.951890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1046/2085] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:38.964698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:38.966264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.042426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51678a19f8f4f7bb62990809729d8b4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.066459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51678a19f8f4f7bb62990809729d8b4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.067646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51678a19f8f4f7bb62990809729d8b4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.081961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51678a19f8f4f7bb62990809729d8b4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.083788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51678a19f8f4f7bb62990809729d8b4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.085835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77e6c9-6cdc-4ff0-a7f4-9a3192021ab3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:39.090343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:39.091142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.091638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.092118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.094294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.102965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1040/2086] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:38 2022] DELETE /identity/v3/users/f51678a19f8f4f7bb62990809729d8b4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:39.110043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:39.111657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:39.125206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1047/2087] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:39.237637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:39.237637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.238826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:39.240156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.273264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b5fe8-7b17-42a4-bf73-54b6b7df2545 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:39.341743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23970b-bc5c-4129-b017-9a1146d2c67c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:39.348522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:39.349158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.349442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.349715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] PATH_INFO: `/v3/users/fe7ab77daeb242248c6d6bf80ce6779c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.351114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7ab77daeb242248c6d6bf80ce6779c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:39.352870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:39.353236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:39.360656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:39.364114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:39.365073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.365460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.366237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] PATH_INFO: `/v3/projects/9c88c47136644d29a80116a1f94fa1c0` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.368353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c88c47136644d29a80116a1f94fa1c0)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:39.371333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:39.439331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7ab77daeb242248c6d6bf80ce6779c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.457134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c88c47136644d29a80116a1f94fa1c0', 'request_id': 'req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.459994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7ab77daeb242248c6d6bf80ce6779c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.460983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7ab77daeb242248c6d6bf80ce6779c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.464992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c88c47136644d29a80116a1f94fa1c0', 'request_id': 'req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.473003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c88c47136644d29a80116a1f94fa1c0', 'request_id': 'req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.473981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7ab77daeb242248c6d6bf80ce6779c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.474799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7ab77daeb242248c6d6bf80ce6779c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.492507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] The token cache is being invalidate because project 9c88c47136644d29a80116a1f94fa1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:39.492908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.494155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3bdd4a9d-9ca6-485b-92f0-0db6bfb6e655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.496155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1041/2088] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:39 2022] DELETE /identity/v3/users/fe7ab77daeb242248c6d6bf80ce6779c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:39.497144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:39.498663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.499748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1048/2089] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:39 2022] DELETE /identity/v3/projects/9c88c47136644d29a80116a1f94fa1c0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:39.502169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.502582 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.503277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.528305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:39.529918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.598333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1049/2090] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:39.602707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:39.604449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.625308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda19d4-3eb6-4c6f-b259-84d1c0cb0f06 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:39.626016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748f6fe-804b-413c-901a-f907f723c1d7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:39.633752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:39.634326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.634609 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.634883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] PATH_INFO: `/v3/projects/afda2a77aa3c48cc946e64ca4ffc1e83` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.636343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afda2a77aa3c48cc946e64ca4ffc1e83)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:39.638260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:39.642711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:39.643731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.643731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.643960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] PATH_INFO: `/v3/users/548057e4dfd348899ff1dfc481c4f1a3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.645115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=548057e4dfd348899ff1dfc481c4f1a3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:39.647054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:39.716258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afda2a77aa3c48cc946e64ca4ffc1e83', 'request_id': 'req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.723495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afda2a77aa3c48cc946e64ca4ffc1e83', 'request_id': 'req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.730303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afda2a77aa3c48cc946e64ca4ffc1e83', 'request_id': 'req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.734453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548057e4dfd348899ff1dfc481c4f1a3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.748433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] The token cache is being invalidate because project afda2a77aa3c48cc946e64ca4ffc1e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:39.748921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.749871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.751064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548057e4dfd348899ff1dfc481c4f1a3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.756442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1042/2091] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:39 2022] DELETE /identity/v3/projects/afda2a77aa3c48cc946e64ca4ffc1e83 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:39.757507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:39.762001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.763389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548057e4dfd348899ff1dfc481c4f1a3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.775474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548057e4dfd348899ff1dfc481c4f1a3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.785595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cae19e1c-851b-48fe-bf57-f3783c7037e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548057e4dfd348899ff1dfc481c4f1a3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:39.787351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:39.798547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1050/2092] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:39 2022] DELETE /identity/v3/users/548057e4dfd348899ff1dfc481c4f1a3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:39.814925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:39.818920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266ecd-455c-4b4d-aaa9-145845adc82d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.820691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266ecd-455c-4b4d-aaa9-145845adc82d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.821187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266ecd-455c-4b4d-aaa9-145845adc82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.871007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d4c59-e91b-4d11-be6e-3c42caa1f450 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:39.875672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4c266ecd-455c-4b4d-aaa9-145845adc82d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:39.878327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4c266ecd-455c-4b4d-aaa9-145845adc82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.931142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:39.931960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.932346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.932720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:39.935334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:39.987149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1051/2093] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:39.995050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144cdbc-6291-464f-838a-18b662995fcc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:39.995227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144cdbc-6291-464f-838a-18b662995fcc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:39.995369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144cdbc-6291-464f-838a-18b662995fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.020647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7144cdbc-6291-464f-838a-18b662995fcc None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:40.023400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7144cdbc-6291-464f-838a-18b662995fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.025077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.028473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.034201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1052/2094] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:40.040861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144cdbc-6291-464f-838a-18b662995fcc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.042711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1043/2095] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:40.065842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144cdbc-6291-464f-838a-18b662995fcc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.073374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.074414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.074910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.075382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.076261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:40.076991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.080579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.095280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1053/2096] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:40.099961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.112114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eddd2-eaae-42a3-8bc7-b8d83d5c9762 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.117359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.118517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.118993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.119461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.120303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.122588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.166148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1044/2097] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:40 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:40.167359 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.183119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523a3da-27bb-426c-a587-0f122cdf089b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.189978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.192720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.284563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81c2a-2cac-4e23-98ed-2c048a554b08 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.289754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.290811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.290811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.291162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] PATH_INFO: `/v3/projects/7800c9caeb0a4fe186a0e9d27a56300c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.292569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7800c9caeb0a4fe186a0e9d27a56300c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.293959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.294643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.295200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.295867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.296830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:40.297331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.298594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.300945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.329910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1054/2098] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:40 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:40.334469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.354922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb493e-ebd8-4314-b54c-d3c84537cf40 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.369876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7800c9caeb0a4fe186a0e9d27a56300c', 'request_id': 'req-216b8bbf-f44b-4e11-8833-4db7dba4235a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.377537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7800c9caeb0a4fe186a0e9d27a56300c', 'request_id': 'req-216b8bbf-f44b-4e11-8833-4db7dba4235a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.395230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7800c9caeb0a4fe186a0e9d27a56300c', 'request_id': 'req-216b8bbf-f44b-4e11-8833-4db7dba4235a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.425556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] The token cache is being invalidate because project 7800c9caeb0a4fe186a0e9d27a56300c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:40.425964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.427372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.433043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1045/2099] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:40 2022] DELETE /identity/v3/projects/7800c9caeb0a4fe186a0e9d27a56300c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:40.434052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.435572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.450563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5005e0-9d98-4f8c-8bd6-89a7b653e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.452570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5005e0-9d98-4f8c-8bd6-89a7b653e1bc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.452865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5005e0-9d98-4f8c-8bd6-89a7b653e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.453171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5005e0-9d98-4f8c-8bd6-89a7b653e1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.455197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5005e0-9d98-4f8c-8bd6-89a7b653e1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.457600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5005e0-9d98-4f8c-8bd6-89a7b653e1bc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.474066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1055/2100] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:40.474066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.474066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.474066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.484211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:40.502510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:40.504721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.513803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8bbf-f44b-4e11-8833-4db7dba4235a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.514499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1056/2101] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:40.519143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.520112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.520548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.520988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] PATH_INFO: `/v3/projects/ab8fa422354844b1b9291d2b2d6d2ea5` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.522924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab8fa422354844b1b9291d2b2d6d2ea5)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.523146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.524595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.525399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.537565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce9a1f-4058-4755-bc56-bd71288c01bf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.542426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe26b1a-5c80-4424-96e1-6c76d0a4ca4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.543127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe26b1a-5c80-4424-96e1-6c76d0a4ca4b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.543481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe26b1a-5c80-4424-96e1-6c76d0a4ca4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.543827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe26b1a-5c80-4424-96e1-6c76d0a4ca4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.544575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe26b1a-5c80-4424-96e1-6c76d0a4ca4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.546250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe26b1a-5c80-4424-96e1-6c76d0a4ca4b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.564583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1057/2102] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:40.567571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.568281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.569190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.598318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab8fa422354844b1b9291d2b2d6d2ea5', 'request_id': 'req-43d53d6b-bad6-438d-8d69-0e824a9d312f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.604520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab8fa422354844b1b9291d2b2d6d2ea5', 'request_id': 'req-43d53d6b-bad6-438d-8d69-0e824a9d312f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.604955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:40.606182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.613932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab8fa422354844b1b9291d2b2d6d2ea5', 'request_id': 'req-43d53d6b-bad6-438d-8d69-0e824a9d312f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.634251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] The token cache is being invalidate because project ab8fa422354844b1b9291d2b2d6d2ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:40.634643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.636261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.639933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1046/2103] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:40 2022] DELETE /identity/v3/projects/ab8fa422354844b1b9291d2b2d6d2ea5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:40.643241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.647544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.674488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:40.692479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1058/2104] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:40.692479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.695163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.713665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b78174-a273-42cd-941d-8ef6b0ddbbad None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.727708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.728981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.730664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.731053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] PATH_INFO: `/v3/users/887f874cd79442b89313b05b3f80f4d5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.732964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=887f874cd79442b89313b05b3f80f4d5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.736951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.784415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53d6b-bad6-438d-8d69-0e824a9d312f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:40.790463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:40.791349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.791791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.792226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] PATH_INFO: `/v3/projects/bcf6a94f240b4397b81af1b5018a5725/users/7cbd742148c94176ab91adcc88f96e3f/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.816775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf6a94f240b4397b81af1b5018a5725, user_id=7cbd742148c94176ab91adcc88f96e3f, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:40.817466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '887f874cd79442b89313b05b3f80f4d5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.820053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:40.831609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '887f874cd79442b89313b05b3f80f4d5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.843151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '887f874cd79442b89313b05b3f80f4d5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.844238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '887f874cd79442b89313b05b3f80f4d5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.851936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1047/2105] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:40 2022] PUT /identity/v3/projects/bcf6a94f240b4397b81af1b5018a5725/users/7cbd742148c94176ab91adcc88f96e3f/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:40.853300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:40.854716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f621fbfb-f1ec-496c-83b4-de1f70bf7bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '887f874cd79442b89313b05b3f80f4d5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:40.863893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1059/2106] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:40 2022] DELETE /identity/v3/users/887f874cd79442b89313b05b3f80f4d5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:40.863893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:40.864155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:40.864624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:40.916612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] MFA Rules not processed for user `7cbd742148c94176ab91adcc88f96e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:40.916612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:40.949384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eff2f-fb62-4a3e-b4cd-d8c07e64157b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.023897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.024538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.024831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.025111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.025808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:41.026133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.027799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.029047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:41.053942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1048/2107] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:05:41.055885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.061336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:41.070973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1060/2108] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:41.079107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.081834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:41.095686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8682a-05c8-45dd-b3ba-6d8eb97d3cc3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.123000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.123622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.124888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.127413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.127999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.145970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.211281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1049/2109] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:41.212825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.225149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742df0e-550e-453e-a5c9-97e1410feeb3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.248594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:41.299292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.300072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.300530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.301123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.321034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90209578-a29a-45ab-a33c-65a649ac70e5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.327271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.328019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.328896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.328896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.329523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.332279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.366612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1050/2110] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:41.367610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.371827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:41.379696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.381194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.384626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057b2fc-571d-4a84-9013-62662a10b087 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.389446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.390208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.390593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.391132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] PATH_INFO: `/v3/projects/6e35e344ed0c49b79a7fc3c2f946ab5f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.392557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e35e344ed0c49b79a7fc3c2f946ab5f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.394687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1061/2111] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:41.394827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.397658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.421673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d150c-6c1d-4446-948d-1696d951f23c service glance] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.430681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.431414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.431783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.432173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.432848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.434576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.447629 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1062/2112] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:41.450483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.455192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e35e344ed0c49b79a7fc3c2f946ab5f', 'request_id': 'req-768caffd-7f31-4e1f-8700-b2ee8d88e411'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:41.462459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c100f-779d-4109-a4fe-61598cfbadfd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.463781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e35e344ed0c49b79a7fc3c2f946ab5f', 'request_id': 'req-768caffd-7f31-4e1f-8700-b2ee8d88e411'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:41.472445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e35e344ed0c49b79a7fc3c2f946ab5f', 'request_id': 'req-768caffd-7f31-4e1f-8700-b2ee8d88e411'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:41.490880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] The token cache is being invalidate because project 6e35e344ed0c49b79a7fc3c2f946ab5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:41.491176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:41.491893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-768caffd-7f31-4e1f-8700-b2ee8d88e411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:41.493446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1051/2113] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:41 2022] DELETE /identity/v3/projects/6e35e344ed0c49b79a7fc3c2f946ab5f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:41.498127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d6de5-95de-47e1-a361-865ca745627d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.498416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d6de5-95de-47e1-a361-865ca745627d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.498696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d6de5-95de-47e1-a361-865ca745627d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.499154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:41.522722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-663d6de5-95de-47e1-a361-865ca745627d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:41.523954 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-663d6de5-95de-47e1-a361-865ca745627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:41.535576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.536221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.536494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.536773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.537411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.539601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.552033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1063/2114] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:41.554473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.557140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:41.570820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcb344-15fb-4a2b-9356-001faef7765a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.575618 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fad534-9afb-4b7c-8e92-452f12184092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.576375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fad534-9afb-4b7c-8e92-452f12184092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.576822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fad534-9afb-4b7c-8e92-452f12184092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.576998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fad534-9afb-4b7c-8e92-452f12184092 admin admin] PATH_INFO: `/v3/projects/349c4a28c984434abd8cd198d1bc6c4f/users/f31b0e4a2995472b8468ec9e71a265a8/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.598177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fad534-9afb-4b7c-8e92-452f12184092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349c4a28c984434abd8cd198d1bc6c4f, user_id=f31b0e4a2995472b8468ec9e71a265a8, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.600174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1052/2115] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:41.601068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fad534-9afb-4b7c-8e92-452f12184092 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.605791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f081d-bf33-4abf-87ee-1257263f7414 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.606181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f081d-bf33-4abf-87ee-1257263f7414 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.606461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f081d-bf33-4abf-87ee-1257263f7414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.617410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1064/2116] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:41 2022] PUT /identity/v3/projects/349c4a28c984434abd8cd198d1bc6c4f/users/f31b0e4a2995472b8468ec9e71a265a8/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:41.628222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dac39-7ca8-40dd-b8ad-33390762ea00 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.628642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dac39-7ca8-40dd-b8ad-33390762ea00 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.628934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dac39-7ca8-40dd-b8ad-33390762ea00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.633054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7a5f081d-bf33-4abf-87ee-1257263f7414 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:41.634259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7a5f081d-bf33-4abf-87ee-1257263f7414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:41.664638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-837dac39-7ca8-40dd-b8ad-33390762ea00 None None] MFA Rules not processed for user `f31b0e4a2995472b8468ec9e71a265a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:41.666703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-837dac39-7ca8-40dd-b8ad-33390762ea00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:41.673419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:41.712540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1053/2117] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:41.716318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f081d-bf33-4abf-87ee-1257263f7414 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.727564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f081d-bf33-4abf-87ee-1257263f7414 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.732644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.733316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.733656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.733989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.735196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:41.735196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.737079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.741625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1065/2118] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:41.746561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dac39-7ca8-40dd-b8ad-33390762ea00 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.751894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1054/2119] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:05:41.755138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.759677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dac39-7ca8-40dd-b8ad-33390762ea00 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.765287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2800f62-d52b-40a1-98f8-7e6c63b19415 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.770838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.771721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.772611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.773224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.774171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.775417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.776653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.777102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.777223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.777652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.778526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.781219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:41.816019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1066/2120] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:41.820190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.820560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1055/2121] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:41.824750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:41.910695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348690cc-347c-4052-9630-28709f7013b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.931823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:41.950245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ad9e9-5af9-49e5-ba50-ac997ed466da admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:41.957172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:41.957909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:41.958254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:41.958595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:41.959696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:41.972383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.001098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1056/2122] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:42.002948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.015115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe757fce-beff-4832-ab2e-801a1bc5e00e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.021101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.022030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.022653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.023063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.024025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.025850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.038033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1057/2123] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:42.041708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.047165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.048374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.048727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.049067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.049772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.053368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.056878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f8d76-ca15-4263-9e36-2a88e32e4eb3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.063984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.065019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.065019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.065431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.066044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.068007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.085017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1058/2124] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:42.085202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1067/2125] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:41 2022] POST /identity/v3/users => generated 298 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:42.086434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.086547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.097379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81074e7-ff6f-49d5-8142-8a0074f5f366 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.097645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59f494-2a67-4480-8d20-910dfef6ab8a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.102761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9d385-edeb-48c0-bbfb-5fb92ee25cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.103117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.103417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9d385-edeb-48c0-bbfb-5fb92ee25cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.103703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9d385-edeb-48c0-bbfb-5fb92ee25cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.103809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.104015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9d385-edeb-48c0-bbfb-5fb92ee25cd5 admin admin] PATH_INFO: `/v3/projects/7ceb5120b019419aa3ace5f177a2f9ca/users/5d4465e4d7ab48c4b3b396bb950a9a04/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.104110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.104351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.105182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.107230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.126017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1059/2126] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:42.127283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.129120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9d385-edeb-48c0-bbfb-5fb92ee25cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ceb5120b019419aa3ace5f177a2f9ca, user_id=5d4465e4d7ab48c4b3b396bb950a9a04, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.134964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9d385-edeb-48c0-bbfb-5fb92ee25cd5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.138652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4763fda-085f-480d-bbcf-366e7738e51e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.144587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.145375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.145798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.145895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] PATH_INFO: `/v3/projects/dc26fffeda4b4488ae8f3e65d79bdd47/users/1767a5192adf490d9ab59f3293b0ce1a/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.156025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1068/2127] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:42 2022] PUT /identity/v3/projects/7ceb5120b019419aa3ace5f177a2f9ca/users/5d4465e4d7ab48c4b3b396bb950a9a04/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:42.170182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e183dac-38b2-4592-bdfc-4258dcbfd17e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.170469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e183dac-38b2-4592-bdfc-4258dcbfd17e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.170740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e183dac-38b2-4592-bdfc-4258dcbfd17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.171732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc26fffeda4b4488ae8f3e65d79bdd47, user_id=1767a5192adf490d9ab59f3293b0ce1a, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.174792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.189588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1060/2128] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:42 2022] PUT /identity/v3/projects/dc26fffeda4b4488ae8f3e65d79bdd47/users/1767a5192adf490d9ab59f3293b0ce1a/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:42.193782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.199700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0e183dac-38b2-4592-bdfc-4258dcbfd17e None None] MFA Rules not processed for user `5d4465e4d7ab48c4b3b396bb950a9a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:42.200945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e183dac-38b2-4592-bdfc-4258dcbfd17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:42.231196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:42.270304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d10005-2045-4fea-af7d-cfe3bf59ff83 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.276202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.276811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.277098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.277372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.277702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1069/2129] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:42.278023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.279638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.282677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e183dac-38b2-4592-bdfc-4258dcbfd17e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.290243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1061/2130] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:42.294225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.295062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e183dac-38b2-4592-bdfc-4258dcbfd17e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.300578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee8f61-34ce-44c0-8aef-cfa7f5fa1edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.301188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8f61-34ce-44c0-8aef-cfa7f5fa1edd admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.301605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8f61-34ce-44c0-8aef-cfa7f5fa1edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.301979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8f61-34ce-44c0-8aef-cfa7f5fa1edd admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.302686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee8f61-34ce-44c0-8aef-cfa7f5fa1edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.304769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee8f61-34ce-44c0-8aef-cfa7f5fa1edd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.305336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ecdcf-fdb7-4a95-82dc-3b47f3a3b4b4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.310715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.311374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.312147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.312147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] PATH_INFO: `/v3/projects/dc26fffeda4b4488ae8f3e65d79bdd47/users/1767a5192adf490d9ab59f3293b0ce1a/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.314528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc26fffeda4b4488ae8f3e65d79bdd47, user_id=1767a5192adf490d9ab59f3293b0ce1a, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.318201 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.331507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1062/2131] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:42 2022] PUT /identity/v3/projects/dc26fffeda4b4488ae8f3e65d79bdd47/users/1767a5192adf490d9ab59f3293b0ce1a/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:42.339092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1070/2132] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:42 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:42.340219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.341961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.342366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.343548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.384888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] MFA Rules not processed for user `1767a5192adf490d9ab59f3293b0ce1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:42.388201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:42.460424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:42.465162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025bf4a-3c4a-409e-92d2-658059bd7643 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.496052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1071/2133] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:42.504820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.506165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:42.534704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.535331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.535628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.535926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.536551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.538506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.560113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1063/2134] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:42 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:42.564345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.568241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c16657-43ee-4f3d-b20f-dae20721bd06 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.571740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.572441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.572788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.573158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.574596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b770410-0be4-4fa3-b43e-a9b6af0469c8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.579516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.580136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.580414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.580688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.581331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.582926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.593037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1064/2135] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:05:42.597850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.608044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92ff3-214f-4df5-8b1b-25fad56340af admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.613875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.614513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.614794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.615318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] PATH_INFO: `/v3/projects/2dd7869bdfac4e76aa682eec6077f3ed/users/29c7ab71f1e84e2e90e068d6298d4029/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.632685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd7869bdfac4e76aa682eec6077f3ed, user_id=29c7ab71f1e84e2e90e068d6298d4029, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.634601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.635365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.636069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.645186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1072/2136] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:42.648490 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1065/2137] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:42 2022] PUT /identity/v3/projects/2dd7869bdfac4e76aa682eec6077f3ed/users/29c7ab71f1e84e2e90e068d6298d4029/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:42.654454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.666024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c02f1-2533-4ec6-889e-edad2b81dbd7 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.747391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7990c-d721-41af-88d2-fac2b7eb44cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.748230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7990c-d721-41af-88d2-fac2b7eb44cb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.748639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7990c-d721-41af-88d2-fac2b7eb44cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.748988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7990c-d721-41af-88d2-fac2b7eb44cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.751446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7990c-d721-41af-88d2-fac2b7eb44cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.753078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7990c-d721-41af-88d2-fac2b7eb44cb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.763693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1073/2138] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:42.768937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.779989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ec978-4f11-4e1e-8ec2-9d236eee76a3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:42.784836 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:42.785642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.785967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.786311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] PATH_INFO: `/v3/projects/2dd7869bdfac4e76aa682eec6077f3ed/users/29c7ab71f1e84e2e90e068d6298d4029/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.788389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd7869bdfac4e76aa682eec6077f3ed, user_id=29c7ab71f1e84e2e90e068d6298d4029, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:42.791158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:42.805589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1066/2139] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:42 2022] PUT /identity/v3/projects/2dd7869bdfac4e76aa682eec6077f3ed/users/29c7ab71f1e84e2e90e068d6298d4029/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:42.816138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:42.816427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:42.816702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:42.846550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] MFA Rules not processed for user `29c7ab71f1e84e2e90e068d6298d4029`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:42.848010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:42.864239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:42.929541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1074/2140] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:42.942550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:42.948766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:43.061103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ce8c6-41b1-4d0e-b442-c69e7a66567f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:43.069656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:43.070809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:43.071731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:43.072374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:43.085815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:43.087263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:43.097809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1067/2141] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:43.184375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:43.185711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:43.199932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0562b0a-d57f-4157-a470-73b5604a351d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:43.204971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:43.205591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:43.205872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:43.206146 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:43.301307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:43.303755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:43.316893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1075/2142] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:49.097444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:49.107413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981fd3b-e7d4-4a24-b651-3e91bccc62ed service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:49.112465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36a9e3-203e-4f4b-ac56-343d30237144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:49.113223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36a9e3-203e-4f4b-ac56-343d30237144 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:49.113623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36a9e3-203e-4f4b-ac56-343d30237144 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:49.113972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36a9e3-203e-4f4b-ac56-343d30237144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:49.245492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36a9e3-203e-4f4b-ac56-343d30237144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:49.247467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36a9e3-203e-4f4b-ac56-343d30237144 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:49.276124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1068/2143] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:49.729200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:49.815053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591c3a9-f97d-4594-9970-11233d96e139 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:49.820308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:49.820951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:49.821236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:49.821512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] PATH_INFO: `/v3/users/5d4465e4d7ab48c4b3b396bb950a9a04` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:49.822764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4465e4d7ab48c4b3b396bb950a9a04)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:49.824764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:49.873794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4465e4d7ab48c4b3b396bb950a9a04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:49.876421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:49.888420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4465e4d7ab48c4b3b396bb950a9a04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:49.895787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4465e4d7ab48c4b3b396bb950a9a04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:49.896450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4465e4d7ab48c4b3b396bb950a9a04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:49.903659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4465e4d7ab48c4b3b396bb950a9a04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:49.910871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1076/2144] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:49 2022] DELETE /identity/v3/users/5d4465e4d7ab48c4b3b396bb950a9a04 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:49.915309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:49.915596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:49.915901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:49.934516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:49.935715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:49.995407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1069/2145] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:50.005426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:50.006817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:50.065276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2563344-13cb-4428-98b6-049ce28fe710 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:50.068835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:50.069526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:50.069843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:50.070035 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:50.081188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:50.082619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:50.092190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1077/2146] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:50.391008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:50.393583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:50.409891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abac2f-750a-4007-ae00-1094bea6534c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:50.415007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:50.415786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:50.416306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:50.416640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] PATH_INFO: `/v3/projects/7ceb5120b019419aa3ace5f177a2f9ca` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:50.417865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ceb5120b019419aa3ace5f177a2f9ca)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:50.419958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:50.480453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:50.492802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ceb5120b019419aa3ace5f177a2f9ca', 'request_id': 'req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.498806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ceb5120b019419aa3ace5f177a2f9ca', 'request_id': 'req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.507000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ceb5120b019419aa3ace5f177a2f9ca', 'request_id': 'req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.530126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] The token cache is being invalidate because project 7ceb5120b019419aa3ace5f177a2f9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:50.530488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.533414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.538437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1070/2147] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:50 2022] DELETE /identity/v3/projects/7ceb5120b019419aa3ace5f177a2f9ca => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:50.539169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:50.555988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa7cc-83a9-45f3-9156-34a682550cfb service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:50.615055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:50.615813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:50.616132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:50.616411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] PATH_INFO: `/v3/users/29c7ab71f1e84e2e90e068d6298d4029` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:50.617609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29c7ab71f1e84e2e90e068d6298d4029)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:50.619555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:50.676678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29c7ab71f1e84e2e90e068d6298d4029'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.690168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29c7ab71f1e84e2e90e068d6298d4029'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.704625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29c7ab71f1e84e2e90e068d6298d4029'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.704625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29c7ab71f1e84e2e90e068d6298d4029'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.705481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:50.711699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29c7ab71f1e84e2e90e068d6298d4029'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:50.719315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1078/2148] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:50 2022] DELETE /identity/v3/users/29c7ab71f1e84e2e90e068d6298d4029 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:50.875391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:50.877733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:50.892604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c374c-1e2c-4db2-8503-b667c2c5ea9a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:50.969276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:50.970537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:50.970882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:50.971218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] PATH_INFO: `/v3/users/6029b853df904de6ad16e209dd237a29` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:50.972534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6029b853df904de6ad16e209dd237a29)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:50.974548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.027680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.029147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.030838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6029b853df904de6ad16e209dd237a29'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.044972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59b6b3-2d67-4b67-aa8d-05677a287547 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.048883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6029b853df904de6ad16e209dd237a29'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.049781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6029b853df904de6ad16e209dd237a29'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.050847 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.051511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.051964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.052302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] PATH_INFO: `/v3/projects/2dd7869bdfac4e76aa682eec6077f3ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.053838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd7869bdfac4e76aa682eec6077f3ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.056004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.060499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6029b853df904de6ad16e209dd237a29'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.061225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4fafa134-57fa-4314-b295-8794cf530db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6029b853df904de6ad16e209dd237a29'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.076589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1071/2149] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:05:50 2022] DELETE /identity/v3/users/6029b853df904de6ad16e209dd237a29 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:51.081294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.081586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.081864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.101911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:51.103074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.108971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd7869bdfac4e76aa682eec6077f3ed', 'request_id': 'req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.115335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd7869bdfac4e76aa682eec6077f3ed', 'request_id': 'req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.121885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd7869bdfac4e76aa682eec6077f3ed', 'request_id': 'req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.134616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] The token cache is being invalidate because project 2dd7869bdfac4e76aa682eec6077f3ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:51.134905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.135527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-de7c6077-7a46-47b6-b837-ce2b8c18fe0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.137059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1079/2150] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:51 2022] DELETE /identity/v3/projects/2dd7869bdfac4e76aa682eec6077f3ed => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:05:51.141903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab628-3103-4ef9-8ad6-f1aacb7d784f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.142292 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab628-3103-4ef9-8ad6-f1aacb7d784f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.142571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab628-3103-4ef9-8ad6-f1aacb7d784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.145118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:51.167468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bb7ab628-3103-4ef9-8ad6-f1aacb7d784f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:51.168745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bb7ab628-3103-4ef9-8ad6-f1aacb7d784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.169943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1072/2151] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:51.179989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1080/2152] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:51.180581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.181986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.188418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e3231-e054-4198-b08f-15df7497354f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.188825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e3231-e054-4198-b08f-15df7497354f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.189259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e3231-e054-4198-b08f-15df7497354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.209004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7a6e3231-e054-4198-b08f-15df7497354f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:51.210207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a6e3231-e054-4198-b08f-15df7497354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.218134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1081/2153] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:51.222919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e3231-e054-4198-b08f-15df7497354f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.232014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e3231-e054-4198-b08f-15df7497354f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.237214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.237804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.238085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.238379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.239031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:51.239362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.241009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.241811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5bf2b-6138-45ad-879f-b07243f84912 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.246242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.246854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.247186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.247399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.253994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1082/2154] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:05:51.258122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.258278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.259562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.269030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd273-202c-4a67-acb9-30d25b5afd5d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.269901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1073/2155] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:51.274688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.275320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.275642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.275951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.276584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.278486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.307680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1083/2156] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:51.312262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.322585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f808bef-a43d-4cbe-9ff5-068e853fc676 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.387560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.388197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.388477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.388752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.389385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.391313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.420918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1074/2157] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:51.425376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.442289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5007d-e907-4285-b13b-c2cc0eaeedc4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.447643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.448338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.448627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.448977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.449720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.451559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.464688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1084/2158] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:51.469055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.481056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840b123-9ae1-4fc6-913a-112b9d4b9279 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.485968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.486656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.486935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.487206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.487872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.489471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.499912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1075/2159] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:51.504102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.515748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a3ec-6c97-4a35-8665-2435c6d72d59 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.521205 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15a3c6-ec41-42bf-a589-8d7fb51a6a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.521870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15a3c6-ec41-42bf-a589-8d7fb51a6a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.522422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15a3c6-ec41-42bf-a589-8d7fb51a6a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.522705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15a3c6-ec41-42bf-a589-8d7fb51a6a24 admin admin] PATH_INFO: `/v3/projects/384c3af9af7b4ea8b7a96d4dc4e8df79/users/e362af3885c744d6bd94ab6e3f75010f/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.538920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15a3c6-ec41-42bf-a589-8d7fb51a6a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384c3af9af7b4ea8b7a96d4dc4e8df79, user_id=e362af3885c744d6bd94ab6e3f75010f, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.541852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15a3c6-ec41-42bf-a589-8d7fb51a6a24 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.556311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.558317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.572732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1085/2160] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:51 2022] PUT /identity/v3/projects/384c3af9af7b4ea8b7a96d4dc4e8df79/users/e362af3885c744d6bd94ab6e3f75010f/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:51.573278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef710e6-0c76-4daf-b007-c91cd72a742d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.575052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.575412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.575764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.617417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] MFA Rules not processed for user `e362af3885c744d6bd94ab6e3f75010f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:51.619190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.649503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.650141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.650428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.650704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] PATH_INFO: `/v3/projects/6079df79fc1b485cb130e81f61dac695` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.651876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6079df79fc1b485cb130e81f61dac695)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.654226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.663091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:51.667650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:51.708597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1086/2161] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:51.716701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.718932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.719710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6079df79fc1b485cb130e81f61dac695', 'request_id': 'req-e7c2639c-9804-4b3a-82c1-c2f601e1beff'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.725966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6079df79fc1b485cb130e81f61dac695', 'request_id': 'req-e7c2639c-9804-4b3a-82c1-c2f601e1beff'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.733075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6079df79fc1b485cb130e81f61dac695', 'request_id': 'req-e7c2639c-9804-4b3a-82c1-c2f601e1beff'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.757545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] The token cache is being invalidate because project 6079df79fc1b485cb130e81f61dac695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:05:51.757849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.758843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e7c2639c-9804-4b3a-82c1-c2f601e1beff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:05:51.764235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1076/2162] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:05:51 2022] DELETE /identity/v3/projects/6079df79fc1b485cb130e81f61dac695 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:51.765796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b617b7-88e8-4802-a545-cfd0caf56429 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.766072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b617b7-88e8-4802-a545-cfd0caf56429 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.766350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b617b7-88e8-4802-a545-cfd0caf56429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.790810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f0b617b7-88e8-4802-a545-cfd0caf56429 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:51.792059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f0b617b7-88e8-4802-a545-cfd0caf56429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.812255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2643c8-7667-46f7-8bc6-c547c001febf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.874624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1077/2163] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:51.887464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd14ac7-52e6-49d5-94c6-da1df3346e72 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.887753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd14ac7-52e6-49d5-94c6-da1df3346e72 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.888190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd14ac7-52e6-49d5-94c6-da1df3346e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.890219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.890984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.891381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.895044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.895044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.923957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6fd14ac7-52e6-49d5-94c6-da1df3346e72 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:51.925765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6fd14ac7-52e6-49d5-94c6-da1df3346e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:51.938381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1078/2164] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:05:51.943385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd14ac7-52e6-49d5-94c6-da1df3346e72 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:51.954136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd14ac7-52e6-49d5-94c6-da1df3346e72 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:51.959940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:51.961080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:51.961581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:51.962890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:51.963974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:05:51.964386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.965580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:51.967354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.969294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:51.990839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1079/2165] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:05:51.994911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.006772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1087/2166] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:05:52.007189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2920a3-b940-4b5c-9a43-4e6807161d75 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.013326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.014090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.014387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.014664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.015305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.018221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.068489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1080/2167] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:51 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:52.073706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.087572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af70e95-a25c-47f6-b2d2-c094f2ffde49 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.151614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:52.185256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.185919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.186254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.186550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.187185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.190890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.216069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1088/2168] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:52 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:52.217326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.228137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee3c52-0710-4b9a-8da2-75e3606b5f12 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.232762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.233444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.233727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.234000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.234701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.237113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.247825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1081/2169] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:52.253087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.265842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbac58-0b7c-4a42-9383-f9b2de1d45be admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.272432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92ea51-73b4-41aa-a3f9-929b210b6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.273034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92ea51-73b4-41aa-a3f9-929b210b6c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.273314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92ea51-73b4-41aa-a3f9-929b210b6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.273587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92ea51-73b4-41aa-a3f9-929b210b6c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.274250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92ea51-73b4-41aa-a3f9-929b210b6c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.275878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92ea51-73b4-41aa-a3f9-929b210b6c54 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.286992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1089/2170] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:52.291159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.303645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbc41e-3497-4910-adb4-f6c669e2110c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.308638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.309289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.309587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.309891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] PATH_INFO: `/v3/projects/4f969d50d33b437ab3bac70f662704cd/users/2b7c85de9cb642d0af66acb72d2816d0/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.328228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f969d50d33b437ab3bac70f662704cd, user_id=2b7c85de9cb642d0af66acb72d2816d0, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.331070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.347092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1082/2171] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:52 2022] PUT /identity/v3/projects/4f969d50d33b437ab3bac70f662704cd/users/2b7c85de9cb642d0af66acb72d2816d0/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:52.357420 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15fa59-550c-4d19-bb48-9c4038a78808 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.357714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15fa59-550c-4d19-bb48-9c4038a78808 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.357995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15fa59-550c-4d19-bb48-9c4038a78808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.400291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6d15fa59-550c-4d19-bb48-9c4038a78808 None None] MFA Rules not processed for user `2b7c85de9cb642d0af66acb72d2816d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:52.401531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d15fa59-550c-4d19-bb48-9c4038a78808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:52.468749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1090/2172] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:52.473760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.494996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:52.538599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade851d1-235b-4209-941a-c2172cdb4d62 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.543347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.543983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.544420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.544707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.545347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.547263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.573591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1083/2173] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:52 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:52.577594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15fa59-550c-4d19-bb48-9c4038a78808 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.588387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15fa59-550c-4d19-bb48-9c4038a78808 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.644903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.645515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.645792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.646065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.646716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.648911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.671738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1091/2174] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:52 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:52.676098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.687123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b33fef-0622-4eb0-9de1-f78a91e3347a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.692066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.692664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.692944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.693222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.693860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.695445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.706946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1084/2175] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:52.711051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.721329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930c03-c99d-433b-89aa-c8a25896e7cd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.725865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12721e-671e-42ec-8d22-d350549fc10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.726453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12721e-671e-42ec-8d22-d350549fc10a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.726731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12721e-671e-42ec-8d22-d350549fc10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.727003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12721e-671e-42ec-8d22-d350549fc10a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.727642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12721e-671e-42ec-8d22-d350549fc10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.729255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12721e-671e-42ec-8d22-d350549fc10a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.738941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1092/2176] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:52.743418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:52.753652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e5918-1598-4c9d-a3c0-f9c91a1ee3e5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:52.758353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:52.758975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.759262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.759539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] PATH_INFO: `/v3/projects/f7b3c7aa75b448cc9b6832ded81a037d/users/0b73a99f9e774251ad43c41737ecde0f/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.774736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b3c7aa75b448cc9b6832ded81a037d, user_id=0b73a99f9e774251ad43c41737ecde0f, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:52.777414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:52.790763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1085/2177] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:52 2022] PUT /identity/v3/projects/f7b3c7aa75b448cc9b6832ded81a037d/users/0b73a99f9e774251ad43c41737ecde0f/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:52.857632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef439506-f090-42c4-b968-71e5bb091f81 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:52.857946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef439506-f090-42c4-b968-71e5bb091f81 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:52.858225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef439506-f090-42c4-b968-71e5bb091f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:52.888702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ef439506-f090-42c4-b968-71e5bb091f81 None None] MFA Rules not processed for user `0b73a99f9e774251ad43c41737ecde0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:52.890206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ef439506-f090-42c4-b968-71e5bb091f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:52.917071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:52.962201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1093/2178] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:52.967251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:53.050016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925058f8-bfbb-42c7-a659-e33d331d0b14 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:53.055150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:53.055939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.056222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.056497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.057151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:53.059254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:53.096856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1086/2179] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:05:52 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:05:53.101145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef439506-f090-42c4-b968-71e5bb091f81 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:53.112427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef439506-f090-42c4-b968-71e5bb091f81 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:53.172782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:53.173333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.173638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.174008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.174648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:53.176653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:53.200153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1094/2180] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:05:53 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:05:53.206164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:53.220131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbeb4c-c44a-4be5-8ab5-c9fe87db64a2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:53.225315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:53.225936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.226218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.226496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.227151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:53.229083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:53.241575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1087/2181] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:53.249986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:53.264017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41c712-abd5-4b09-bcd5-b6db8ff59716 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:53.271793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:53.272772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.273054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.273332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] PATH_INFO: `/v3/projects/e5afe3fa7ccd4fcf9738eb891abf753b/users/c242def8d8814bdc8fd14151e059de44/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.300657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5afe3fa7ccd4fcf9738eb891abf753b, user_id=c242def8d8814bdc8fd14151e059de44, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:53.303587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:53.326299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1095/2182] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:53 2022] PUT /identity/v3/projects/e5afe3fa7ccd4fcf9738eb891abf753b/users/c242def8d8814bdc8fd14151e059de44/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:53.327618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:53.342633 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc30f-2d8a-4de2-a3bc-d05bf8955490 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:53.424273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:53.442453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e92c3-e616-41e7-bfd9-d3a36790a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:53.443237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e92c3-e616-41e7-bfd9-d3a36790a8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.443522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e92c3-e616-41e7-bfd9-d3a36790a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.443807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e92c3-e616-41e7-bfd9-d3a36790a8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.444487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e92c3-e616-41e7-bfd9-d3a36790a8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:53.446145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e92c3-e616-41e7-bfd9-d3a36790a8a4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:53.457120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1088/2183] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:05:53.461268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:53.471964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f5af8-5c66-45dc-8b59-25fc82a69af0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:53.477482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:53.478137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.478421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.478700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] PATH_INFO: `/v3/projects/e5afe3fa7ccd4fcf9738eb891abf753b/users/c242def8d8814bdc8fd14151e059de44/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.480660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5afe3fa7ccd4fcf9738eb891abf753b, user_id=c242def8d8814bdc8fd14151e059de44, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:53.483298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:53.494235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1096/2184] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:05:53 2022] PUT /identity/v3/projects/e5afe3fa7ccd4fcf9738eb891abf753b/users/c242def8d8814bdc8fd14151e059de44/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:05:53.504965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40518f8f-cb10-4c20-916f-973ddb3e4dd1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.505452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40518f8f-cb10-4c20-916f-973ddb3e4dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.505568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40518f8f-cb10-4c20-916f-973ddb3e4dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.534771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-40518f8f-cb10-4c20-916f-973ddb3e4dd1 None None] MFA Rules not processed for user `c242def8d8814bdc8fd14151e059de44`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:05:53.536007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-40518f8f-cb10-4c20-916f-973ddb3e4dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:53.607603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1089/2185] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:05:53.619665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:53.621373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:05:53.680325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26a46f-30ae-48ad-9757-8f31a8a78ed7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:53.683920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:53.684519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:53.684796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:53.685067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:53.717481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:05:53.746523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:53.747984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:53.757350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1097/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:55.329187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40518f8f-cb10-4c20-916f-973ddb3e4dd1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:55.340346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40518f8f-cb10-4c20-916f-973ddb3e4dd1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:55.343904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:55.344516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:55.345304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:55.346215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:55.411417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:55.412892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:55.421873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1090/2187] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:05:56.140481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:05:56.150095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5670a3-fff2-43fc-9726-84fd7a2738ab service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:05:56.153743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d486558-41d0-404b-9a65-c915576227b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:05:56.154344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d486558-41d0-404b-9a65-c915576227b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:05:56.154628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d486558-41d0-404b-9a65-c915576227b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:05:56.154903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d486558-41d0-404b-9a65-c915576227b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:05:56.165123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d486558-41d0-404b-9a65-c915576227b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:05:56.166593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d486558-41d0-404b-9a65-c915576227b3 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:05:56.176943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1098/2188] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:05:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:01.912571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:01.914074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:01.979520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a8412-4a49-4113-b506-31ab3cb3cf95 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:01.984117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:01.984738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:01.985022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:01.985793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] PATH_INFO: `/v3/users/f31b0e4a2995472b8468ec9e71a265a8` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:01.987001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31b0e4a2995472b8468ec9e71a265a8)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:01.988984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:01.993594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:02.036765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31b0e4a2995472b8468ec9e71a265a8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.049112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31b0e4a2995472b8468ec9e71a265a8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.049785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31b0e4a2995472b8468ec9e71a265a8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.056948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31b0e4a2995472b8468ec9e71a265a8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.057643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31b0e4a2995472b8468ec9e71a265a8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.070491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1091/2189] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:01 2022] DELETE /identity/v3/users/f31b0e4a2995472b8468ec9e71a265a8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:02.076440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:02.076964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:02.076964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:02.097112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:02.098308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:02.155647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1099/2190] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:02.165662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:02.167131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:02.226023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ebb0d-120f-4165-a149-783c1145656b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:02.229965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:02.230570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:02.230849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:02.231119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:02.241853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:02.243226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:02.252918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1092/2191] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:02.527975 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:02.535627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:02.561968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad2804-21db-436a-9843-ddec46f0084d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:02.576804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:02.577545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:02.577726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:02.578025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] PATH_INFO: `/v3/projects/349c4a28c984434abd8cd198d1bc6c4f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:02.580499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349c4a28c984434abd8cd198d1bc6c4f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:02.582472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:02.650691 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349c4a28c984434abd8cd198d1bc6c4f', 'request_id': 'req-e190375a-8b7f-44d3-9a84-fdf1c9257576'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.656431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349c4a28c984434abd8cd198d1bc6c4f', 'request_id': 'req-e190375a-8b7f-44d3-9a84-fdf1c9257576'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.663514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349c4a28c984434abd8cd198d1bc6c4f', 'request_id': 'req-e190375a-8b7f-44d3-9a84-fdf1c9257576'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.683183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] The token cache is being invalidate because project 349c4a28c984434abd8cd198d1bc6c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:02.683479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.684181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.686133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1100/2192] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:02 2022] DELETE /identity/v3/projects/349c4a28c984434abd8cd198d1bc6c4f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:02.690124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:02.700244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c62b4-5438-48b3-baf7-a35fe8ab32f0 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:02.764576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:02.765653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:02.765932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:02.766209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] PATH_INFO: `/v3/users/1767a5192adf490d9ab59f3293b0ce1a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:02.768813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1767a5192adf490d9ab59f3293b0ce1a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:02.771677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:02.816055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:02.841070 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1767a5192adf490d9ab59f3293b0ce1a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.856282 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1767a5192adf490d9ab59f3293b0ce1a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.857043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1767a5192adf490d9ab59f3293b0ce1a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.869150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1767a5192adf490d9ab59f3293b0ce1a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.869795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2fc57910-d36b-4dba-9d95-6f3b4e08a09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1767a5192adf490d9ab59f3293b0ce1a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:02.885648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1093/2193] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:02 2022] DELETE /identity/v3/users/1767a5192adf490d9ab59f3293b0ce1a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:03.197077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.198654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:03.218350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190375a-8b7f-44d3-9a84-fdf1c9257576 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.294613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:03.320733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:03.321386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.321665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.321963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] PATH_INFO: `/v3/projects/dc26fffeda4b4488ae8f3e65d79bdd47` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.323326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc26fffeda4b4488ae8f3e65d79bdd47)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:03.333664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:03.417950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc26fffeda4b4488ae8f3e65d79bdd47', 'request_id': 'req-99a1bc87-8992-42eb-b7c7-e32c8ac54112'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.425149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc26fffeda4b4488ae8f3e65d79bdd47', 'request_id': 'req-99a1bc87-8992-42eb-b7c7-e32c8ac54112'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.432461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc26fffeda4b4488ae8f3e65d79bdd47', 'request_id': 'req-99a1bc87-8992-42eb-b7c7-e32c8ac54112'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.450734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] The token cache is being invalidate because project dc26fffeda4b4488ae8f3e65d79bdd47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:03.451038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.451759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-99a1bc87-8992-42eb-b7c7-e32c8ac54112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.453700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1101/2194] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:03 2022] DELETE /identity/v3/projects/dc26fffeda4b4488ae8f3e65d79bdd47 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:03.459811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf91c-c921-4a9d-80bc-86d285c5f2c7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.460301 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf91c-c921-4a9d-80bc-86d285c5f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.460430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbf91c-c921-4a9d-80bc-86d285c5f2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.484200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1abbf91c-c921-4a9d-80bc-86d285c5f2c7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:03.485980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1abbf91c-c921-4a9d-80bc-86d285c5f2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:03.553829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1094/2195] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:03.563124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfd2bb-5990-4497-81a1-68398494abbb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.564678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfd2bb-5990-4497-81a1-68398494abbb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.565029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfd2bb-5990-4497-81a1-68398494abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.591129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ddfd2bb-5990-4497-81a1-68398494abbb None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:03.592362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ddfd2bb-5990-4497-81a1-68398494abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:03.601857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1102/2196] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:03.606433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf91c-c921-4a9d-80bc-86d285c5f2c7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.616897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbf91c-c921-4a9d-80bc-86d285c5f2c7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.621939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:03.622581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.622899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.623172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.623970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:03.624274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:03.625905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:03.644181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1095/2197] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:06:03.645238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfd2bb-5990-4497-81a1-68398494abbb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.659408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfd2bb-5990-4497-81a1-68398494abbb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.665369 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:03.666362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.666646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.666920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.667573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:03.670143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:03.698637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.700415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:03.714425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1103/2198] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:03 2022] POST /identity/v3/projects => generated 399 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:03.715528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f211ec-61cb-4b4f-aa6c-8f3a9fe1d33f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.718876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.802751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:03.803526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.803821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.804152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] PATH_INFO: `/v3/users/e362af3885c744d6bd94ab6e3f75010f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.805268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e362af3885c744d6bd94ab6e3f75010f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:03.807396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:03.809261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049e83-56ca-48b9-917f-a0cca8ffb42d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.816019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:03.816786 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.817194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.817564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.818295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:03.822031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:03.846801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1104/2199] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:03 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:03.851112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.862256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:03.866326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95cccc8-356a-4d2b-8b36-0cf83ab7dd63 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.874977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:03.876735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.877349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.877794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.878236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e362af3885c744d6bd94ab6e3f75010f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.878655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:03.880990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:03.893989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e362af3885c744d6bd94ab6e3f75010f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.894899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e362af3885c744d6bd94ab6e3f75010f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.896197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1105/2200] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:03.900483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.902309 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:03.908433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e362af3885c744d6bd94ab6e3f75010f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.909307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-adc29ee7-51d3-478a-8c20-f851d59d024b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e362af3885c744d6bd94ab6e3f75010f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:03.919283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a76c0-d046-4bac-a4d0-9db9dde628ce admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.924884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:03.925770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1096/2201] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:03 2022] DELETE /identity/v3/users/e362af3885c744d6bd94ab6e3f75010f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:03.926268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.926653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.927005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.927727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:03.930643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:03.930974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:03.931266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:03.931539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:03.944437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1106/2202] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:03.948779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:03.959695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:03.960918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:03.961565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5ee3-7b40-4406-81d6-4ac8eff5a1fc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:03.973784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:04.040567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0229443-8d7a-4084-8a74-5f5712851623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:04.041240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229443-8d7a-4084-8a74-5f5712851623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.041617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229443-8d7a-4084-8a74-5f5712851623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.041798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229443-8d7a-4084-8a74-5f5712851623 admin admin] PATH_INFO: `/v3/projects/a03d257fca2c465ba601295f9bffe870/users/52d7ecc6fa2d40e8a54e3c311ea47d50/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.042003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1097/2203] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:04.056148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:04.057901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.059573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0229443-8d7a-4084-8a74-5f5712851623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03d257fca2c465ba601295f9bffe870, user_id=52d7ecc6fa2d40e8a54e3c311ea47d50, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:04.062504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0229443-8d7a-4084-8a74-5f5712851623 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:04.085497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1107/2204] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:03 2022] PUT /identity/v3/projects/a03d257fca2c465ba601295f9bffe870/users/52d7ecc6fa2d40e8a54e3c311ea47d50/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:04.097449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.097953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.098139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.134709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] MFA Rules not processed for user `52d7ecc6fa2d40e8a54e3c311ea47d50`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:04.136223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.147966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9965b-6b17-44f0-9e90-97f0d43db77e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:04.216064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:04.216676 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.216960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.217242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.222596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1108/2205] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:04.236621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:04.240655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.243793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:04.304956 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:04.306426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:04.319207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1098/2206] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:04.320002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70095b9f-6d20-47da-bb39-54060803cda7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:04.324153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b750b-9edc-47ef-86b9-c3932a8c6d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:04.325081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b750b-9edc-47ef-86b9-c3932a8c6d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.325321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b750b-9edc-47ef-86b9-c3932a8c6d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.325676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b750b-9edc-47ef-86b9-c3932a8c6d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.338279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b750b-9edc-47ef-86b9-c3932a8c6d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:04.340336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b750b-9edc-47ef-86b9-c3932a8c6d16 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:04.350661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1109/2207] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:04.420302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:04.422574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.447402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4c131-f82b-4076-a2ee-a652c5f02184 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:04.452170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:04.452793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.453072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.453346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] PATH_INFO: `/v3/users/2b7c85de9cb642d0af66acb72d2816d0` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.454715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7c85de9cb642d0af66acb72d2816d0)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:04.456749 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:04.519193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7c85de9cb642d0af66acb72d2816d0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.539784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7c85de9cb642d0af66acb72d2816d0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.540429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7c85de9cb642d0af66acb72d2816d0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.550280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7c85de9cb642d0af66acb72d2816d0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.550990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7c85de9cb642d0af66acb72d2816d0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.573596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1099/2208] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:04 2022] DELETE /identity/v3/users/2b7c85de9cb642d0af66acb72d2816d0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:04.575236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.575531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.575809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.599299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:04.600817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.626311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:04.683562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1110/2209] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:04.693696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:04.695585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.696501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:04.701300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.728231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf8737-a9ff-4cd9-b67f-c3da7b5da58a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:04.737387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:04.737387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.737387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.737387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] PATH_INFO: `/v3/projects/384c3af9af7b4ea8b7a96d4dc4e8df79` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.738243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=384c3af9af7b4ea8b7a96d4dc4e8df79)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:04.740855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:04.780209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc979-74d1-432f-8d7d-6e61dfa84cbb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:04.786050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dfc0c-4dd6-4189-b4ee-dd48595dea80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:04.786795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dfc0c-4dd6-4189-b4ee-dd48595dea80 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.787126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dfc0c-4dd6-4189-b4ee-dd48595dea80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.787435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dfc0c-4dd6-4189-b4ee-dd48595dea80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.798467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dfc0c-4dd6-4189-b4ee-dd48595dea80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:04.800560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dfc0c-4dd6-4189-b4ee-dd48595dea80 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:04.811797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1111/2210] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:04.815687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384c3af9af7b4ea8b7a96d4dc4e8df79', 'request_id': 'req-43ef0b90-cd48-4500-8754-1d65e890ac0c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.824893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384c3af9af7b4ea8b7a96d4dc4e8df79', 'request_id': 'req-43ef0b90-cd48-4500-8754-1d65e890ac0c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.837179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384c3af9af7b4ea8b7a96d4dc4e8df79', 'request_id': 'req-43ef0b90-cd48-4500-8754-1d65e890ac0c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.854404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] The token cache is being invalidate because project 384c3af9af7b4ea8b7a96d4dc4e8df79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:04.854753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.855936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43ef0b90-cd48-4500-8754-1d65e890ac0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:04.861798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1100/2211] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:04 2022] DELETE /identity/v3/projects/384c3af9af7b4ea8b7a96d4dc4e8df79 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:04.863596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3d2d-40d7-4c66-9249-c0081c6ea3d7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:04.863958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3d2d-40d7-4c66-9249-c0081c6ea3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:04.864190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3d2d-40d7-4c66-9249-c0081c6ea3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:04.895671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d41e3d2d-40d7-4c66-9249-c0081c6ea3d7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:04.897972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d41e3d2d-40d7-4c66-9249-c0081c6ea3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:04.993715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1112/2212] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:05.002240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bd457-ed38-4202-b6d5-acd041a7573d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.002674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bd457-ed38-4202-b6d5-acd041a7573d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.002958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bd457-ed38-4202-b6d5-acd041a7573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.026922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-569bd457-ed38-4202-b6d5-acd041a7573d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:05.028812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-569bd457-ed38-4202-b6d5-acd041a7573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:05.038867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1101/2213] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:05.043371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3d2d-40d7-4c66-9249-c0081c6ea3d7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.053413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3d2d-40d7-4c66-9249-c0081c6ea3d7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.058103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.058703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.058980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.059247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.059925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:05.060232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.061821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.075359 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1113/2214] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:06:05.080552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bd457-ed38-4202-b6d5-acd041a7573d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.090169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bd457-ed38-4202-b6d5-acd041a7573d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.095292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.095869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.096156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.096447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.097115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.098998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.103520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:05.130136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1102/2215] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:05 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:05.134315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.145213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44a22e-cb29-429e-9639-a17f20cbad23 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.149906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.152444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:05.199697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:05.207498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.208175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.208462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.208727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.209372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.211298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.237315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1114/2216] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:05 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:05.241246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.247458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cf078-524f-49fe-8a23-fcea31ff0102 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.251993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4944d8-c8f6-42c2-a1f8-e8b5295cd37e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.253361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.253995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.254278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.254588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] PATH_INFO: `/v3/projects/4f969d50d33b437ab3bac70f662704cd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.255707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f969d50d33b437ab3bac70f662704cd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.257294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.257757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.258128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.258700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.259196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.260038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.262027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.277887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1115/2217] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:05.281720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.293836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29616f-28b4-4052-8ef5-47f9f47e55fd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.301376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.302990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.303288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.303665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.304373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.306737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.317993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f969d50d33b437ab3bac70f662704cd', 'request_id': 'req-0d886c10-c66e-47fd-8311-0d0f4e576df2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.321651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1116/2218] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:05.322755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.324315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f969d50d33b437ab3bac70f662704cd', 'request_id': 'req-0d886c10-c66e-47fd-8311-0d0f4e576df2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.333109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f969d50d33b437ab3bac70f662704cd', 'request_id': 'req-0d886c10-c66e-47fd-8311-0d0f4e576df2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.336656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83aec-5041-4548-bee2-64581375e1ae admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.349170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] The token cache is being invalidate because project 4f969d50d33b437ab3bac70f662704cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:05.349466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.350123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.352003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1103/2219] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:05 2022] DELETE /identity/v3/projects/4f969d50d33b437ab3bac70f662704cd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:05.355531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.356895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:05.426124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b978ba-9014-44f4-8544-93e5c75a1da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.427228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b978ba-9014-44f4-8544-93e5c75a1da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.427664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b978ba-9014-44f4-8544-93e5c75a1da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.427983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b978ba-9014-44f4-8544-93e5c75a1da1 admin admin] PATH_INFO: `/v3/projects/e3aa5b736bb64bfe9fdf06cd170710a6/users/beb377cb89434d7196f69e883b3d8dd5/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.451028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d886c10-c66e-47fd-8311-0d0f4e576df2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.455714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b978ba-9014-44f4-8544-93e5c75a1da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3aa5b736bb64bfe9fdf06cd170710a6, user_id=beb377cb89434d7196f69e883b3d8dd5, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.457071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.457629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.457911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.458185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] PATH_INFO: `/v3/users/0b73a99f9e774251ad43c41737ecde0f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.460263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b978ba-9014-44f4-8544-93e5c75a1da1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.462208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b73a99f9e774251ad43c41737ecde0f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.466290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.476475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1117/2220] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:05 2022] PUT /identity/v3/projects/e3aa5b736bb64bfe9fdf06cd170710a6/users/beb377cb89434d7196f69e883b3d8dd5/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:05.488194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.488376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.488652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.488876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:05.524272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] MFA Rules not processed for user `beb377cb89434d7196f69e883b3d8dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:05.525702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:05.535166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b73a99f9e774251ad43c41737ecde0f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.554796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b73a99f9e774251ad43c41737ecde0f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.555557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b73a99f9e774251ad43c41737ecde0f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.565714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b73a99f9e774251ad43c41737ecde0f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.566794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b73a99f9e774251ad43c41737ecde0f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:05.580447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1104/2221] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:05 2022] DELETE /identity/v3/users/0b73a99f9e774251ad43c41737ecde0f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:05.610097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1118/2222] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:05.637471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.639330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:05.717851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216d1c-6dc3-4beb-bba1-c246d5725da8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:05.722028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:05.722636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:05.722914 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:05.723184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:05.815399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:05.817912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:05.832979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1105/2223] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:05.889801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:05.891289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:05.954454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:05.988758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e0f5f-ad94-405c-9ef2-55d88618203e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:06.097567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:06.101085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:06.101381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:06.101661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] PATH_INFO: `/v3/projects/f7b3c7aa75b448cc9b6832ded81a037d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:06.102911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b3c7aa75b448cc9b6832ded81a037d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:06.104957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:06.166789 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b3c7aa75b448cc9b6832ded81a037d', 'request_id': 'req-a0f9e086-00eb-4a39-8573-b257774fd58f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.184539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b3c7aa75b448cc9b6832ded81a037d', 'request_id': 'req-a0f9e086-00eb-4a39-8573-b257774fd58f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.199701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b3c7aa75b448cc9b6832ded81a037d', 'request_id': 'req-a0f9e086-00eb-4a39-8573-b257774fd58f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.220557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] The token cache is being invalidate because project f7b3c7aa75b448cc9b6832ded81a037d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:06.221388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.222076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.224285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1119/2224] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:05 2022] DELETE /identity/v3/projects/f7b3c7aa75b448cc9b6832ded81a037d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:06.228230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:06.251134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbc056-3615-4863-bc65-e3e36409a749 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:06.314163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:06.330937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:06.330937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:06.330937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:06.331416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] PATH_INFO: `/v3/users/c242def8d8814bdc8fd14151e059de44` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:06.332640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c242def8d8814bdc8fd14151e059de44)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:06.334707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:06.383659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c242def8d8814bdc8fd14151e059de44'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.402555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c242def8d8814bdc8fd14151e059de44'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.403413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c242def8d8814bdc8fd14151e059de44'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.412148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c242def8d8814bdc8fd14151e059de44'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.412802 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c242def8d8814bdc8fd14151e059de44'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.430889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1106/2225] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:06 2022] DELETE /identity/v3/users/c242def8d8814bdc8fd14151e059de44 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:06.793208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:06.794663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:06.808669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9e086-00eb-4a39-8573-b257774fd58f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:06.869883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:06.870568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:06.870852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:06.871132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] PATH_INFO: `/v3/projects/e5afe3fa7ccd4fcf9738eb891abf753b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:06.872314 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5afe3fa7ccd4fcf9738eb891abf753b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:06.874231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:06.891373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:06.922437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5afe3fa7ccd4fcf9738eb891abf753b', 'request_id': 'req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.928513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5afe3fa7ccd4fcf9738eb891abf753b', 'request_id': 'req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.934729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5afe3fa7ccd4fcf9738eb891abf753b', 'request_id': 'req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.947041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] The token cache is being invalidate because project e5afe3fa7ccd4fcf9738eb891abf753b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:06.947330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.947974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:06.949446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1120/2226] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:06 2022] DELETE /identity/v3/projects/e5afe3fa7ccd4fcf9738eb891abf753b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:14.794234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:14.795632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:14.808788 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae30821-ba86-4214-9eaa-8d0bc81d2f5e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:14.865397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:14.866036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:14.866327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:14.866599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] PATH_INFO: `/v3/users/52d7ecc6fa2d40e8a54e3c311ea47d50` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:14.867718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d7ecc6fa2d40e8a54e3c311ea47d50)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:14.869699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:14.876522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:14.878993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:14.891216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed2f4-a9b9-4de0-a90d-0bd9ab291c38 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:14.896494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:14.897361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:14.897697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:14.897962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] PATH_INFO: `/v3/users/beb377cb89434d7196f69e883b3d8dd5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:14.899109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb377cb89434d7196f69e883b3d8dd5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:14.901058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:14.919595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d7ecc6fa2d40e8a54e3c311ea47d50'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.937525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d7ecc6fa2d40e8a54e3c311ea47d50'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.938583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d7ecc6fa2d40e8a54e3c311ea47d50'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.945004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:14.952232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d7ecc6fa2d40e8a54e3c311ea47d50'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.953069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6530265c-afde-4c4f-abae-d3707c36e8a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d7ecc6fa2d40e8a54e3c311ea47d50'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.970981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb377cb89434d7196f69e883b3d8dd5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.975910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1107/2227] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:14 2022] DELETE /identity/v3/users/52d7ecc6fa2d40e8a54e3c311ea47d50 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:14.979331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:14.979634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:14.980119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:14.984840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb377cb89434d7196f69e883b3d8dd5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.993962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb377cb89434d7196f69e883b3d8dd5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:14.994626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb377cb89434d7196f69e883b3d8dd5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.003059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48e81740-0991-4a1a-b5ca-8b1e2c61ba99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb377cb89434d7196f69e883b3d8dd5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.005266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:15.006260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.009271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1121/2228] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:14 2022] DELETE /identity/v3/users/beb377cb89434d7196f69e883b3d8dd5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:15.013294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.013577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.013846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.034288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:15.035510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.072345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1108/2229] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:15.083280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:15.084838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.108950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1122/2230] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:15.119200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:15.121191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.147864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:15.151977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde55b-cc4a-48b8-9e67-af3efb54ced1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:15.186547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0f6bb-9899-49e0-8a4e-6b0399f465da None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:15.191014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:15.191616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.191928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.192206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.201131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:15.202555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:15.203317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:15.203963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.204251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.204530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.212588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1123/2231] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:15.214374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:15.215830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:15.225385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1109/2232] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:15.598604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:15.601594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.616117 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5e509-a7f3-4adb-84b9-86e85463602b service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:15.621559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:15.622200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.622498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.622753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] PATH_INFO: `/v3/projects/e3aa5b736bb64bfe9fdf06cd170710a6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.624867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3aa5b736bb64bfe9fdf06cd170710a6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:15.627749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:15.652830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:15.654603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.670516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e45a0c-1847-48e3-914e-4e5fb0000b41 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:15.684994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3aa5b736bb64bfe9fdf06cd170710a6', 'request_id': 'req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.692927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3aa5b736bb64bfe9fdf06cd170710a6', 'request_id': 'req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.699592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3aa5b736bb64bfe9fdf06cd170710a6', 'request_id': 'req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.714478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] The token cache is being invalidate because project e3aa5b736bb64bfe9fdf06cd170710a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:15.714811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.715433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dfdf1f4f-dedb-4ff7-9b64-b18fdeb19031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.716967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1124/2233] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:15 2022] DELETE /identity/v3/projects/e3aa5b736bb64bfe9fdf06cd170710a6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:15.721409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:15.721583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02f8f-e8ab-4e70-b2b1-4df93a3a460b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.721829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02f8f-e8ab-4e70-b2b1-4df93a3a460b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.722106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02f8f-e8ab-4e70-b2b1-4df93a3a460b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.738116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:15.738603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.738874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.739147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] PATH_INFO: `/v3/projects/a03d257fca2c465ba601295f9bffe870` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.740401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03d257fca2c465ba601295f9bffe870)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:15.742353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:15.744945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0bd02f8f-e8ab-4e70-b2b1-4df93a3a460b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:15.746185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0bd02f8f-e8ab-4e70-b2b1-4df93a3a460b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.756884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1125/2234] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:15.764413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6be59-61f9-4fc0-92e7-91caa4611a9b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.764783 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6be59-61f9-4fc0-92e7-91caa4611a9b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.765175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6be59-61f9-4fc0-92e7-91caa4611a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.786123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b3e6be59-61f9-4fc0-92e7-91caa4611a9b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:15.787384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b3e6be59-61f9-4fc0-92e7-91caa4611a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:15.798923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03d257fca2c465ba601295f9bffe870', 'request_id': 'req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.804408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03d257fca2c465ba601295f9bffe870', 'request_id': 'req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.810791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03d257fca2c465ba601295f9bffe870', 'request_id': 'req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.823514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] The token cache is being invalidate because project a03d257fca2c465ba601295f9bffe870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:15.823804 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.824537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:15.826071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1110/2235] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:15 2022] DELETE /identity/v3/projects/a03d257fca2c465ba601295f9bffe870 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:15.856805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1126/2236] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:15.861018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:15.873116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2b8e9-4cbb-42b8-a442-646e8d306cb9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:15.879376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:15.880033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.880443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.880583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.881243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:15.881547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:15.883188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:15.902791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1111/2237] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:06:15.903958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6be59-61f9-4fc0-92e7-91caa4611a9b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:15.914494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6be59-61f9-4fc0-92e7-91caa4611a9b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:15.919266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:15.919925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:15.920203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:15.920476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:15.921135 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:15.923084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:15.959872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1127/2238] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:15 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:15.963696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:15.976167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0f0fa-504d-48e6-a35c-810d0ba3c302 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:16.051152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:16.051754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:16.052075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:16.052346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:16.052966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:16.054974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:16.078101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1112/2239] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:15 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:16.082414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:16.092891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc2fa8-3b96-47bc-b875-c1ca1955a2df admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:16.098282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:16.098886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:16.099167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:16.099439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:16.100126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:16.101728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:16.115891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1128/2240] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:16.116995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:16.127442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aea75-69c2-4ad4-b510-4d0e45c87c2c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:16.132096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba446a-74f3-46dc-9b17-d9c9e4bb485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:16.132690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba446a-74f3-46dc-9b17-d9c9e4bb485b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:16.132970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba446a-74f3-46dc-9b17-d9c9e4bb485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:16.133236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba446a-74f3-46dc-9b17-d9c9e4bb485b admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:16.133872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba446a-74f3-46dc-9b17-d9c9e4bb485b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:16.135451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba446a-74f3-46dc-9b17-d9c9e4bb485b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:16.145938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1113/2241] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:16.151234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:16.157483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:16.162048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11fcaf-e30d-447f-829b-1e3803cd6081 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:16.168906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:16.169735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:16.170376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:16.171033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] PATH_INFO: `/v3/projects/fb12bffabd0d43c88d6fd3047d0d2262/users/d88236ac3ad643d6af862ba735a79077/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:16.186583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb12bffabd0d43c88d6fd3047d0d2262, user_id=d88236ac3ad643d6af862ba735a79077, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:16.189374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:16.202241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1129/2242] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:16 2022] PUT /identity/v3/projects/fb12bffabd0d43c88d6fd3047d0d2262/users/d88236ac3ad643d6af862ba735a79077/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:16.214174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:16.214461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:16.214741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:16.242980 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] MFA Rules not processed for user `d88236ac3ad643d6af862ba735a79077`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:16.244239 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:16.275394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:16.313232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1114/2243] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:16.326048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:16.327788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:16.385140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dbe81-618b-4c28-9715-0a5857ce0d77 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:16.388811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:16.389412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:16.389689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:16.389960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:16.398941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:16.400393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:16.409736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1130/2244] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:24.334035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:24.335454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:24.407202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fc699-d5f5-47fc-abaf-b6fef63ced58 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:24.415364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:24.416114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:24.416488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:24.416856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] PATH_INFO: `/v3/users/02930aebe31e450f9fc989af369bf319` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:24.418429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02930aebe31e450f9fc989af369bf319)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:24.420310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:24.474506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02930aebe31e450f9fc989af369bf319'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.487107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02930aebe31e450f9fc989af369bf319'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.487822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02930aebe31e450f9fc989af369bf319'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.496949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02930aebe31e450f9fc989af369bf319'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.498155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-613a44bd-f96c-4275-8f7a-23099e5c98f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02930aebe31e450f9fc989af369bf319'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.520205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1115/2245] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:24 2022] DELETE /identity/v3/users/02930aebe31e450f9fc989af369bf319 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:24.703608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:24.717272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88659a-8b98-48c6-9776-6fc520751be7 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:24.778008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:24.778643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:24.778919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:24.779197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] PATH_INFO: `/v3/projects/aaa6c50978eb46ecba4c88562ede965b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:24.780330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa6c50978eb46ecba4c88562ede965b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:24.782251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:24.833328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa6c50978eb46ecba4c88562ede965b', 'request_id': 'req-e53fb7bd-414c-473d-8e72-b01b495a7708'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.839089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa6c50978eb46ecba4c88562ede965b', 'request_id': 'req-e53fb7bd-414c-473d-8e72-b01b495a7708'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.847779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa6c50978eb46ecba4c88562ede965b', 'request_id': 'req-e53fb7bd-414c-473d-8e72-b01b495a7708'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.860930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] The token cache is being invalidate because project aaa6c50978eb46ecba4c88562ede965b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:24.861678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.862332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:24.864482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1131/2246] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:24 2022] DELETE /identity/v3/projects/aaa6c50978eb46ecba4c88562ede965b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:24.869520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd750f0a-a960-47a3-a82a-119c9b3dbbf2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:24.869737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd750f0a-a960-47a3-a82a-119c9b3dbbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:24.870008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd750f0a-a960-47a3-a82a-119c9b3dbbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:24.892221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fd750f0a-a960-47a3-a82a-119c9b3dbbf2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:24.894020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fd750f0a-a960-47a3-a82a-119c9b3dbbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:24.898694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:24.900046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:24.918621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:24.952352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:24.973978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1116/2247] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:24.981907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d67b2-d78c-4e96-8b93-4f965e9b2284 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:24.982459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d67b2-d78c-4e96-8b93-4f965e9b2284 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:24.982941 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d67b2-d78c-4e96-8b93-4f965e9b2284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:24.989987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fb7bd-414c-473d-8e72-b01b495a7708 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:24.995044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:24.995388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:24.995671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:24.995979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] PATH_INFO: `/v3/users/7cbd742148c94176ab91adcc88f96e3f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:24.997152 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbd742148c94176ab91adcc88f96e3f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:24.999068 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.008602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-713d67b2-d78c-4e96-8b93-4f965e9b2284 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:25.009935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-713d67b2-d78c-4e96-8b93-4f965e9b2284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:25.020360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1117/2248] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:25.025161 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d67b2-d78c-4e96-8b93-4f965e9b2284 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.036385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d67b2-d78c-4e96-8b93-4f965e9b2284 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.041984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.042585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.042864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.043142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.043779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:25.044111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.047447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.055656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbd742148c94176ab91adcc88f96e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.060503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1118/2249] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:06:25.065150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.070462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbd742148c94176ab91adcc88f96e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.075178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6dd1a-f2dc-4fde-b74d-b951e1b23ee4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.080228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbd742148c94176ab91adcc88f96e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.081039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbd742148c94176ab91adcc88f96e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.082344 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.083019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.083289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.084039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.084606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.086863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.089203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c072388-d861-4482-9a8c-8676f788c63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbd742148c94176ab91adcc88f96e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.096952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1132/2250] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:24 2022] DELETE /identity/v3/users/7cbd742148c94176ab91adcc88f96e3f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:25.101042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.101323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.101603 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.122735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1119/2251] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:25 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:25.123949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.125917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:25.127721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:25.128978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:25.202558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c3236-93a1-47f7-873c-3bae2bb02967 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.209199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1133/2252] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:25.209351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.209351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.209351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.209351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.209652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.211474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.215903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.217441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:25.235510 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1120/2253] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:25 2022] POST /identity/v3/users => generated 281 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:25.240179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.250095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132ad8-d273-4ac6-9bc8-7372491d48d4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.256974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.257655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.258012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.258585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.259061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.260777 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.262365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:25.272753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1121/2254] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:25.276649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.296978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa41111-fa39-41a1-b2f0-ec659ce4b577 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.297607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec53a9-f2a9-4e4b-844a-0df24ae5f718 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.301835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d292c9-01e6-41f7-906f-ce95b45c9cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.302240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.302449 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d292c9-01e6-41f7-906f-ce95b45c9cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.302731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d292c9-01e6-41f7-906f-ce95b45c9cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.302849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.303001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d292c9-01e6-41f7-906f-ce95b45c9cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.303105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.303356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.304030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.305696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.313197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d292c9-01e6-41f7-906f-ce95b45c9cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.314849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d292c9-01e6-41f7-906f-ce95b45c9cc9 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.317211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1122/2255] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:25.321483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.326094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1134/2256] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:25.332586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c9e05-bdc0-4dee-acc0-ce5795d0648f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.337406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.338122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.338348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.338618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] PATH_INFO: `/v3/projects/109ca05a36004d38a190864d9563755b/users/342af20d37d84ce7b9dcbf05de8cf96b/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.356046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=109ca05a36004d38a190864d9563755b, user_id=342af20d37d84ce7b9dcbf05de8cf96b, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.358782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.378217 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1123/2257] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:25 2022] PUT /identity/v3/projects/109ca05a36004d38a190864d9563755b/users/342af20d37d84ce7b9dcbf05de8cf96b/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:25.393920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557beeac-f5df-4547-8918-d60f1bb242e5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.393920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557beeac-f5df-4547-8918-d60f1bb242e5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.393920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557beeac-f5df-4547-8918-d60f1bb242e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.427598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-557beeac-f5df-4547-8918-d60f1bb242e5 None None] MFA Rules not processed for user `342af20d37d84ce7b9dcbf05de8cf96b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:25.429184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-557beeac-f5df-4547-8918-d60f1bb242e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:25.506900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1135/2258] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:25.508406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.575033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:25.588373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a301d9b-e4c0-4149-abfd-4500bcc2f170 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.594861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.595984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.596505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.597003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.598107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.600384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.660722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557beeac-f5df-4547-8918-d60f1bb242e5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.666650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1124/2259] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:25 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:25.691989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557beeac-f5df-4547-8918-d60f1bb242e5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.759223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.760757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:25.799141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.799762 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.800134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.800412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.801048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.816093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.849982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1136/2260] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:25 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:25.856894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.866543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf273f-aa54-411e-a894-417103fc2eb0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.872773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.873416 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.873522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dc04d-eff1-4f84-b488-2c993b03fcc4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.873694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.873965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] PATH_INFO: `/v3/projects/bcf6a94f240b4397b81af1b5018a5725` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.875127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcf6a94f240b4397b81af1b5018a5725)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.877796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.882705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.882705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.882705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.882705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.883138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.884168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.899940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1137/2261] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:25.901244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.915668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a71d9-c919-4b11-84e0-b7138b3276b2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:25.923770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:25.924978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.925235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.925568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] PATH_INFO: `/v3/projects/44d318e0298f4be28d979ff2c863e8b2/users/1bb7c8b129f540f29a641004a4f551d2/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.934620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcf6a94f240b4397b81af1b5018a5725', 'request_id': 'req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.941160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcf6a94f240b4397b81af1b5018a5725', 'request_id': 'req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.942337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d318e0298f4be28d979ff2c863e8b2, user_id=1bb7c8b129f540f29a641004a4f551d2, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:25.945131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:25.947967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcf6a94f240b4397b81af1b5018a5725', 'request_id': 'req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.964208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1138/2262] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:25 2022] PUT /identity/v3/projects/44d318e0298f4be28d979ff2c863e8b2/users/1bb7c8b129f540f29a641004a4f551d2/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:25.965242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:25.969067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] The token cache is being invalidate because project bcf6a94f240b4397b81af1b5018a5725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:25.969428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.970138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bc1aa98a-b9e1-4e0c-92fe-575bc5d2a773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:25.972331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1125/2263] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:25 2022] DELETE /identity/v3/projects/bcf6a94f240b4397b81af1b5018a5725 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:25.977051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab3d9-008d-452a-ad7e-3b2772e1803e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:25.977333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab3d9-008d-452a-ad7e-3b2772e1803e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:25.977603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab3d9-008d-452a-ad7e-3b2772e1803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:25.992092 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:25.998086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-960ab3d9-008d-452a-ad7e-3b2772e1803e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:25.999284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-960ab3d9-008d-452a-ad7e-3b2772e1803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:26.040907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8da93-c1e6-4a3f-bd35-072b1eae27f3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.046207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.046928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.047316 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.047693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.048755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.050900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.062907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1139/2264] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:26.067201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.068983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:26.074266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1126/2265] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:26.083337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795c90-b1f8-4e76-82a2-62a07051228f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.083751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795c90-b1f8-4e76-82a2-62a07051228f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.084007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795c90-b1f8-4e76-82a2-62a07051228f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.089796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0f8c-1a90-432b-8b5d-449f84304ccf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.095907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f0401-34c6-49f5-b0b9-6ea88c454506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.096671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f0401-34c6-49f5-b0b9-6ea88c454506 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.097147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f0401-34c6-49f5-b0b9-6ea88c454506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.097486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f0401-34c6-49f5-b0b9-6ea88c454506 admin admin] PATH_INFO: `/v3/projects/44d318e0298f4be28d979ff2c863e8b2/users/1bb7c8b129f540f29a641004a4f551d2/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.099460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f0401-34c6-49f5-b0b9-6ea88c454506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d318e0298f4be28d979ff2c863e8b2, user_id=1bb7c8b129f540f29a641004a4f551d2, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.102218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f0401-34c6-49f5-b0b9-6ea88c454506 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.104592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bc795c90-b1f8-4e76-82a2-62a07051228f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:26.105661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bc795c90-b1f8-4e76-82a2-62a07051228f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:26.113786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1127/2266] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:26.116115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1140/2267] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:26 2022] PUT /identity/v3/projects/44d318e0298f4be28d979ff2c863e8b2/users/1bb7c8b129f540f29a641004a4f551d2/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:26.126208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795c90-b1f8-4e76-82a2-62a07051228f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.126208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:26.129500 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.130027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.130372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.134069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795c90-b1f8-4e76-82a2-62a07051228f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.168513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] MFA Rules not processed for user `1bb7c8b129f540f29a641004a4f551d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:26.169735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:26.202122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.202859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.203604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.203907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.204682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:26.205774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.207488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.225958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1128/2268] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:06:26.227399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.238064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc5035-3a45-4acb-ae60-b44ba856a4d1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.242697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.243285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.243661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.243975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.244605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.246541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.248168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1141/2269] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:26.259845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.261261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:26.283905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1129/2270] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:26 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:26.283905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.336976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5eb73a-4b77-4200-9f3d-47169f739d0f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.347699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fec612-5bb5-461a-86cf-65dac380c39e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.352714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.353311 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.353619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.353889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.354517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.356508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.383227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1130/2271] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:26 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:26.387562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.388650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:26.401424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac87b1-b001-4964-a164-f15ad7e861b3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.401591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811427-9070-4d14-a48d-8db2a76ec756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.404318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811427-9070-4d14-a48d-8db2a76ec756 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.404602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811427-9070-4d14-a48d-8db2a76ec756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.404881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811427-9070-4d14-a48d-8db2a76ec756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.407062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.407766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.408790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.409141 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.409866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.411539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.423189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1131/2272] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:26.427480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.437109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2618be0-9242-450f-8a9e-60596ce601b3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.443259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.443897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.444144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.444414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.445076 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.446660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.462290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1132/2273] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:26.463242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.473938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dae7f3-9ea2-480d-9e7e-26206df59185 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.478850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.479489 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.479925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.480198 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] PATH_INFO: `/v3/projects/6861ba7333684225b0e72933031463cd/users/643fecf195ff46a7a5e6d4e35c438813/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.484914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811427-9070-4d14-a48d-8db2a76ec756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.487120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811427-9070-4d14-a48d-8db2a76ec756 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.496715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6861ba7333684225b0e72933031463cd, user_id=643fecf195ff46a7a5e6d4e35c438813, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.497734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1142/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:26.501425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.519243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1133/2275] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:26 2022] PUT /identity/v3/projects/6861ba7333684225b0e72933031463cd/users/643fecf195ff46a7a5e6d4e35c438813/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:26.531376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f600-73bf-4967-8a14-dda319613cbd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.531657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f600-73bf-4967-8a14-dda319613cbd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.531973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f600-73bf-4967-8a14-dda319613cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.569259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f888f600-73bf-4967-8a14-dda319613cbd None None] MFA Rules not processed for user `643fecf195ff46a7a5e6d4e35c438813`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:26.570833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f888f600-73bf-4967-8a14-dda319613cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:26.664769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1143/2276] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:26.666744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.690454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:26.740979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a8f7d-75d5-49bd-b787-983a55c2dcfb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.746031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.746808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.747190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.747617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.748379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.750509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.782071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1134/2277] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:26 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:26.786851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888f600-73bf-4967-8a14-dda319613cbd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.801991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888f600-73bf-4967-8a14-dda319613cbd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.866650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.867258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.867536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.867805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.868512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.870526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.894427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1144/2278] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:26 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:26.899641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.911124 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2617c2b-02e9-4e72-9acf-38bff59d2b25 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.919385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.920441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.920883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.921960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.922759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.925604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.940315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1135/2279] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:26.947322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.958614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05447ea3-c9ae-47c7-a44e-fc0bb6289598 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.969243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:26.970170 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:26.970986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:26.971263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:26.971943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:26.974295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:26.983033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.988215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1145/2280] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:26.993193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:26.995257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6234c25-a901-49d8-b296-1107a7f5d36b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:26.999039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:27.005203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.005904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.006184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.006426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f88371-dc74-4552-b4bf-f615c6ad87cc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.006524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] PATH_INFO: `/v3/users/342af20d37d84ce7b9dcbf05de8cf96b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.008126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=342af20d37d84ce7b9dcbf05de8cf96b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.010696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.013375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395dbb6-57dc-4a90-a34d-22ef9eb4da69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.014132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395dbb6-57dc-4a90-a34d-22ef9eb4da69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.014532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395dbb6-57dc-4a90-a34d-22ef9eb4da69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.014905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395dbb6-57dc-4a90-a34d-22ef9eb4da69 admin admin] PATH_INFO: `/v3/projects/f10c2d3561ec40e6b6199c25c33e3273/users/ae9996e3871c40a188c4486ca9776764/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.031856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395dbb6-57dc-4a90-a34d-22ef9eb4da69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10c2d3561ec40e6b6199c25c33e3273, user_id=ae9996e3871c40a188c4486ca9776764, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.035123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395dbb6-57dc-4a90-a34d-22ef9eb4da69 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.059520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1146/2281] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:26 2022] PUT /identity/v3/projects/f10c2d3561ec40e6b6199c25c33e3273/users/ae9996e3871c40a188c4486ca9776764/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:27.071640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.072507 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.073030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.092916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342af20d37d84ce7b9dcbf05de8cf96b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.107334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] MFA Rules not processed for user `ae9996e3871c40a188c4486ca9776764`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:27.108024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342af20d37d84ce7b9dcbf05de8cf96b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.108578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.108737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342af20d37d84ce7b9dcbf05de8cf96b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.118259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342af20d37d84ce7b9dcbf05de8cf96b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.119250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b28f8ee-8181-4800-b8b8-98920dfe3d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342af20d37d84ce7b9dcbf05de8cf96b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.134343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1136/2282] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:26 2022] DELETE /identity/v3/users/342af20d37d84ce7b9dcbf05de8cf96b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:27.138898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.139288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.139558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.159505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:27.161269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.180388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1147/2283] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:27.184995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.186353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.222274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1137/2284] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:27.233577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.234716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.242932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:27.260143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8545b28-08b4-4dac-ba85-65237c141de5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.264720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.265347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.265623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.265893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.266525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.268492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.295687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1148/2285] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:27 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:27.300436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.300997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fee1c-a7be-4d84-a522-317dac0e9f20 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.355024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.355729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:27.355888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.355888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.356175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.369647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6e8f-8dac-43f2-8ddf-00beab1ad897 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.374722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.375327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.375608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.375980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.377222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.379137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.401128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1149/2286] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:27 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:27.405196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.416221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c82aa-bd9a-4f17-ad6f-488783402cb2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.420834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.421417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.421723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.422007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.422641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.422757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.424685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.425051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.434291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1138/2287] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:27.435597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1150/2288] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:27.440578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.449823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af999e9-faa9-4f5b-8729-08211fee405e service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.454469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.455410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.455696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.456012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] PATH_INFO: `/v3/projects/2458b6a3de3146d1979ca09397709b99/users/f02d1b4a67854d65a9135e1fc126759a/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.471091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2458b6a3de3146d1979ca09397709b99, user_id=f02d1b4a67854d65a9135e1fc126759a, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.473865 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.488366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1139/2289] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:27 2022] PUT /identity/v3/projects/2458b6a3de3146d1979ca09397709b99/users/f02d1b4a67854d65a9135e1fc126759a/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:27.493017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.504403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39369d3-f7fc-443a-b217-06c063c3845f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.563788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1aef1-b219-4874-bd30-f4355c349f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.564943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1aef1-b219-4874-bd30-f4355c349f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.565220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1aef1-b219-4874-bd30-f4355c349f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.565490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1aef1-b219-4874-bd30-f4355c349f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.566148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1aef1-b219-4874-bd30-f4355c349f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.567741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1aef1-b219-4874-bd30-f4355c349f2a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.578004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1151/2290] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:27.583286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.594023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9abc-fa85-4a5a-a2bc-bf53a42f0d2e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.599059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.599746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.600066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.600337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] PATH_INFO: `/v3/projects/2458b6a3de3146d1979ca09397709b99/users/f02d1b4a67854d65a9135e1fc126759a/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.602378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2458b6a3de3146d1979ca09397709b99, user_id=f02d1b4a67854d65a9135e1fc126759a, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.605091 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.616298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1140/2291] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:27 2022] PUT /identity/v3/projects/2458b6a3de3146d1979ca09397709b99/users/f02d1b4a67854d65a9135e1fc126759a/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:27.627063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.628032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.628326 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.658303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] MFA Rules not processed for user `f02d1b4a67854d65a9135e1fc126759a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:27.660172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.710604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.713364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.737062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1152/2292] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:27.748446 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.750412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.758602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:27.790918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71f826-d5f5-4088-bc46-7ca6104c757c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.796195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.796836 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.797123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.797520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] PATH_INFO: `/v3/projects/109ca05a36004d38a190864d9563755b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.798725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] RBAC: Authorizing `identity:delete_project(project_id=109ca05a36004d38a190864d9563755b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.800683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.817351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a40e8-0bba-41ce-804a-a75a2921ec81 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.821044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.821658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.821977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.822202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.823989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.832869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:27.851276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '109ca05a36004d38a190864d9563755b', 'request_id': 'req-0906e36a-d885-4f4a-99b2-711211bff5af'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.858395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '109ca05a36004d38a190864d9563755b', 'request_id': 'req-0906e36a-d885-4f4a-99b2-711211bff5af'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.865983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '109ca05a36004d38a190864d9563755b', 'request_id': 'req-0906e36a-d885-4f4a-99b2-711211bff5af'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.879683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] The token cache is being invalidate because project 109ca05a36004d38a190864d9563755b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:27.880090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.880770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:27.882296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1141/2293] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:27 2022] DELETE /identity/v3/projects/109ca05a36004d38a190864d9563755b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:27.886267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:27.887691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:27.904755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.906291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:27.916811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1153/2294] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:27.970294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906e36a-d885-4f4a-99b2-711211bff5af admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:27.975066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:27.975693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:27.976004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:27.976276 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] PATH_INFO: `/v3/users/1bb7c8b129f540f29a641004a4f551d2` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:27.977407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb7c8b129f540f29a641004a4f551d2)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:27.979313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:28.029310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb7c8b129f540f29a641004a4f551d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.041470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb7c8b129f540f29a641004a4f551d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.042165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb7c8b129f540f29a641004a4f551d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.043808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:28.050782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb7c8b129f540f29a641004a4f551d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.051779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e6f89dc-9f86-4634-9b9c-4c9aed88aa12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb7c8b129f540f29a641004a4f551d2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.067330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1142/2295] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:27 2022] DELETE /identity/v3/users/1bb7c8b129f540f29a641004a4f551d2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:28.462038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:28.475081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a99a97-b00d-4016-b718-c51e339694d7 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:28.541798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:28.542438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:28.542738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:28.543005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] PATH_INFO: `/v3/projects/44d318e0298f4be28d979ff2c863e8b2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:28.544169 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d318e0298f4be28d979ff2c863e8b2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:28.546128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:28.601470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d318e0298f4be28d979ff2c863e8b2', 'request_id': 'req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.608710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d318e0298f4be28d979ff2c863e8b2', 'request_id': 'req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.616590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d318e0298f4be28d979ff2c863e8b2', 'request_id': 'req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.630926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] The token cache is being invalidate because project 44d318e0298f4be28d979ff2c863e8b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:28.631217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.631864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-59aaed57-4d5b-4fe9-8595-2c96ba1a0d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:28.635884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1154/2296] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:28 2022] DELETE /identity/v3/projects/44d318e0298f4be28d979ff2c863e8b2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:28.640772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3ee50-ae96-4643-8101-e0aa886da2e9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:28.641281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3ee50-ae96-4643-8101-e0aa886da2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:28.641377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3ee50-ae96-4643-8101-e0aa886da2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:28.662924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f7f3ee50-ae96-4643-8101-e0aa886da2e9 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:28.664192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f7f3ee50-ae96-4643-8101-e0aa886da2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:28.726302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1143/2297] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:28.734484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbdc84-7806-441d-9d71-ede9c40c857d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:28.734769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbdc84-7806-441d-9d71-ede9c40c857d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:28.735075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbdc84-7806-441d-9d71-ede9c40c857d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:28.756116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3bbbdc84-7806-441d-9d71-ede9c40c857d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:28.757251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3bbbdc84-7806-441d-9d71-ede9c40c857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:28.768968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1155/2298] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:28.770690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3ee50-ae96-4643-8101-e0aa886da2e9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:28.781107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3ee50-ae96-4643-8101-e0aa886da2e9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:28.786625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:28.787226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:28.787508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:28.787778 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:28.788466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:28.788990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:28.790904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:28.803064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1144/2299] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:06:28.807455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbdc84-7806-441d-9d71-ede9c40c857d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:28.817034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbdc84-7806-441d-9d71-ede9c40c857d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:28.821674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:28.822254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:28.822533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:28.822804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:28.823436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:28.825379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:28.853065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1156/2300] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:28 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:28.857259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:28.867255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6768b-42fb-4959-b226-b2e02d6b20d2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:28.924576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:28.925175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:28.925459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:28.925735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:28.926367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:28.928306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:28.953806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1145/2301] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:28 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:28.955489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:28.966769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8818e-a5fb-4789-9ae1-e59be22b2392 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:28.972324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:28.972925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:28.973201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:28.973468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:28.974104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:28.975683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:28.977957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:28.987755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1157/2302] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:28.991536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.002725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27c81a-fa8d-4641-86c2-5d0efa9a2b39 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.007891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302529f-84de-42c4-a9e1-3c0cc165d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.008464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302529f-84de-42c4-a9e1-3c0cc165d3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.008727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302529f-84de-42c4-a9e1-3c0cc165d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.009002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302529f-84de-42c4-a9e1-3c0cc165d3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.009666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302529f-84de-42c4-a9e1-3c0cc165d3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.011306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302529f-84de-42c4-a9e1-3c0cc165d3a8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.026384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1146/2303] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:29.027536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.038581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255723-7fd7-4f54-9cef-18aae9743702 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.044714 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.045667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.046000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.046411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] PATH_INFO: `/v3/projects/ac69d63139324e65b2b15e50ec7146e4/users/b188611fa9f84df180529436f89fc746/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.063411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac69d63139324e65b2b15e50ec7146e4, user_id=b188611fa9f84df180529436f89fc746, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.066731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.080749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1158/2304] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:29 2022] PUT /identity/v3/projects/ac69d63139324e65b2b15e50ec7146e4/users/b188611fa9f84df180529436f89fc746/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:29.092317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df3dc-0fe3-422f-9fe1-d4f0a55e81b2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.092596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df3dc-0fe3-422f-9fe1-d4f0a55e81b2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.092864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df3dc-0fe3-422f-9fe1-d4f0a55e81b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.120808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-af2df3dc-0fe3-422f-9fe1-d4f0a55e81b2 None None] MFA Rules not processed for user `b188611fa9f84df180529436f89fc746`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:29.122040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-af2df3dc-0fe3-422f-9fe1-d4f0a55e81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:29.161610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:29.189787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1147/2305] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:29.194607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.259776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1fdc-9674-46d5-8f14-5cfea008fc88 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.265347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.265347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.265727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.265727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.266530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.269415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.301552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1159/2306] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:29 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:29.307200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df3dc-0fe3-422f-9fe1-d4f0a55e81b2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.343378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df3dc-0fe3-422f-9fe1-d4f0a55e81b2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.431393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.432119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.432474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.432827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.433572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.435561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.465025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1148/2307] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:29 2022] POST /identity/v3/users => generated 301 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:29.466143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.480572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55f15c-4d47-405e-88c5-fffdd2644981 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.485708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.486956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.487103 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.487372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.492248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.493997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.506626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1160/2308] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:29.511029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.521747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90269c95-1446-4153-9679-114d97eda526 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.526996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.527643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.527955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.528227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] PATH_INFO: `/v3/projects/6e1d9770a05b4531907c076c9433d05a/users/a7b0165735d74eeda063e21b0fbd0e12/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.530177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.531927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:29.543491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d9770a05b4531907c076c9433d05a, user_id=a7b0165735d74eeda063e21b0fbd0e12, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.546190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.559709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1149/2309] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:29 2022] PUT /identity/v3/projects/6e1d9770a05b4531907c076c9433d05a/users/a7b0165735d74eeda063e21b0fbd0e12/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:29.563737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.591553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a73ba0-aac8-4588-8f33-31a5b606f41e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.628639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37179f11-d56f-4418-982a-8dc0f28c2f0c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.633411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.634010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.634292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.634556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.635190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.636796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.641490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.642683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.643172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.643448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.645255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:29.647646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1150/2310] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:29.652074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.662097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc429-7af4-4236-ac0b-427b7ff335a6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.666659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:29.666826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323190f-5006-44d2-b479-37cb33206d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.667304 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323190f-5006-44d2-b479-37cb33206d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.667576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323190f-5006-44d2-b479-37cb33206d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.667877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323190f-5006-44d2-b479-37cb33206d90 admin admin] PATH_INFO: `/v3/projects/6e1d9770a05b4531907c076c9433d05a/users/a7b0165735d74eeda063e21b0fbd0e12/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.669935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323190f-5006-44d2-b479-37cb33206d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d9770a05b4531907c076c9433d05a, user_id=a7b0165735d74eeda063e21b0fbd0e12, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.672708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323190f-5006-44d2-b479-37cb33206d90 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.682961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1151/2311] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:29 2022] PUT /identity/v3/projects/6e1d9770a05b4531907c076c9433d05a/users/a7b0165735d74eeda063e21b0fbd0e12/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:29.693453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e34e7-bdd3-4553-9066-f48a03147fab None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.693736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e34e7-bdd3-4553-9066-f48a03147fab None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.694008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e34e7-bdd3-4553-9066-f48a03147fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.716779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.718371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.722742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-360e34e7-bdd3-4553-9066-f48a03147fab None None] MFA Rules not processed for user `a7b0165735d74eeda063e21b0fbd0e12`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:29.724025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-360e34e7-bdd3-4553-9066-f48a03147fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:29.728771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1161/2312] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:29.737328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:29.795973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1152/2313] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:29.806877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:29.808598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:29.820788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f182c-4849-4f85-be1a-f2dbf59af78e service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:29.881856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:29.882669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:29.883014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:29.883358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:29.893747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:29.895249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:29.905281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1162/2314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:30.042656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e34e7-bdd3-4553-9066-f48a03147fab None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:30.099535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:30.101071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:30.124318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e34e7-bdd3-4553-9066-f48a03147fab None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:30.131083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:30.132424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.132780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.133127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] PATH_INFO: `/v3/users/b188611fa9f84df180529436f89fc746` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.134828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b188611fa9f84df180529436f89fc746)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:30.136898 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:30.182762 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38201b61-03c9-4bc0-9f45-3d3f54ba5bfc service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:30.187547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:30.188207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.188484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.188762 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] PATH_INFO: `/v3/users/d88236ac3ad643d6af862ba735a79077` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.190662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88236ac3ad643d6af862ba735a79077)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:30.192636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:30.198835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b188611fa9f84df180529436f89fc746'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.214820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b188611fa9f84df180529436f89fc746'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.215661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b188611fa9f84df180529436f89fc746'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.224084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b188611fa9f84df180529436f89fc746'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.224707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5159fc56-24db-4701-9ee6-4b3009356f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b188611fa9f84df180529436f89fc746'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.246106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1153/2315] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:30 2022] DELETE /identity/v3/users/b188611fa9f84df180529436f89fc746 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:30.248376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88236ac3ad643d6af862ba735a79077'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.250577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.251232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.252579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.260761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:30.262166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88236ac3ad643d6af862ba735a79077'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.275203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88236ac3ad643d6af862ba735a79077'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.275987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88236ac3ad643d6af862ba735a79077'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.278177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:30.279458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:30.283229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-57557d29-03d8-48d1-9e5e-b2d57c538e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88236ac3ad643d6af862ba735a79077'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:30.296688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1163/2316] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:30 2022] DELETE /identity/v3/users/d88236ac3ad643d6af862ba735a79077 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:30.296688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.296688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.296688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.315215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:30.317006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:30.320377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:30.359244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1154/2317] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:30.360120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:30.361540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:30.384388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1164/2318] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:30.385475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:30.393660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:30.423446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0693cce-6f46-4159-b44e-fa35f2f358ba None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:30.458162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1811d-309b-4784-b96c-b15cc8aa41d8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:30.461969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:30.462566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.462848 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.463240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.473310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:30.474742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:30.475752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:30.476383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.476662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.476932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.478655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:30.484206 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1165/2319] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:30.484999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:30.493282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11b330-21e0-4f05-a679-049d40757151 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:30.497125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:30.498288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.498288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.498288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.508645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:30.510090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:30.524797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1166/2320] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:30.555191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:30.557214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:30.570174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1155/2321] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:30.930305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:30.932245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:30.948096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725004ea-4d67-4a5c-80d9-f0958855e24d service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:30.952940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:30.953625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:30.954080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:30.954403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] PATH_INFO: `/v3/projects/ac69d63139324e65b2b15e50ec7146e4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:30.955768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac69d63139324e65b2b15e50ec7146e4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:30.957910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.003472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.005087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.009787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac69d63139324e65b2b15e50ec7146e4', 'request_id': 'req-06cdef66-7462-44e3-b40e-5d34993f2ec7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.017266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac69d63139324e65b2b15e50ec7146e4', 'request_id': 'req-06cdef66-7462-44e3-b40e-5d34993f2ec7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.020590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9e31-4c3d-4163-818d-a827910879d5 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.027628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac69d63139324e65b2b15e50ec7146e4', 'request_id': 'req-06cdef66-7462-44e3-b40e-5d34993f2ec7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.030585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:31.043360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] The token cache is being invalidate because project ac69d63139324e65b2b15e50ec7146e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:31.043730 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.044454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.046038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1167/2322] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:30 2022] DELETE /identity/v3/projects/ac69d63139324e65b2b15e50ec7146e4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:31.050753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.052108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.066037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:31.091426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.092093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.092375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.092649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] PATH_INFO: `/v3/projects/fb12bffabd0d43c88d6fd3047d0d2262` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.093825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb12bffabd0d43c88d6fd3047d0d2262)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.096074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.120703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdef66-7462-44e3-b40e-5d34993f2ec7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.125750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.126434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.127248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.127521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] PATH_INFO: `/v3/users/a7b0165735d74eeda063e21b0fbd0e12` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.128713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b0165735d74eeda063e21b0fbd0e12)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.130637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.143279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb12bffabd0d43c88d6fd3047d0d2262', 'request_id': 'req-5c03960b-fef5-4e8f-afae-9a27bfa85da3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.149104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb12bffabd0d43c88d6fd3047d0d2262', 'request_id': 'req-5c03960b-fef5-4e8f-afae-9a27bfa85da3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.155979 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb12bffabd0d43c88d6fd3047d0d2262', 'request_id': 'req-5c03960b-fef5-4e8f-afae-9a27bfa85da3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.178805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] The token cache is being invalidate because project fb12bffabd0d43c88d6fd3047d0d2262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:31.179099 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.179729 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5c03960b-fef5-4e8f-afae-9a27bfa85da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.181474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1156/2323] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:31 2022] DELETE /identity/v3/projects/fb12bffabd0d43c88d6fd3047d0d2262 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:31.185531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b0165735d74eeda063e21b0fbd0e12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.185816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10552c31-5897-4f24-96fc-1fda853655e6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.186093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10552c31-5897-4f24-96fc-1fda853655e6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.186363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10552c31-5897-4f24-96fc-1fda853655e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.199203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b0165735d74eeda063e21b0fbd0e12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.208328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b0165735d74eeda063e21b0fbd0e12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.208784 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-10552c31-5897-4f24-96fc-1fda853655e6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:31.208979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b0165735d74eeda063e21b0fbd0e12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.210011 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-10552c31-5897-4f24-96fc-1fda853655e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.217328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f8a0c83f-fcfe-4931-a144-15f052a3ebb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b0165735d74eeda063e21b0fbd0e12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.223698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1168/2324] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:31 2022] DELETE /identity/v3/users/a7b0165735d74eeda063e21b0fbd0e12 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:31.274506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1157/2325] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:31.282577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeecd6b-ab89-4bd8-9120-387c228883db None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.283803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeecd6b-ab89-4bd8-9120-387c228883db None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.284310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeecd6b-ab89-4bd8-9120-387c228883db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.306083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2aeecd6b-ab89-4bd8-9120-387c228883db None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:31.307396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2aeecd6b-ab89-4bd8-9120-387c228883db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.372082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1169/2326] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:31.374467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10552c31-5897-4f24-96fc-1fda853655e6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.389101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10552c31-5897-4f24-96fc-1fda853655e6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.395342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.396125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.396410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.396686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.397606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:31.397915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.399562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.413255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1158/2327] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:06:31.417607 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeecd6b-ab89-4bd8-9120-387c228883db None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.427450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeecd6b-ab89-4bd8-9120-387c228883db None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.432075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.432670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.432948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.433219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.433842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.435784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.446979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:31.469048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1170/2328] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:31.470793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.480995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8132fd-bd02-461b-8a1d-cbac555ec58e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.520370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.522542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.539522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:31.549308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.550542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.550753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.551026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.551667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.554318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.577361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1159/2329] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:31.581634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.591415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e486e-0f2b-43b9-8893-55170eaf2e5c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.596538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.597123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.597397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.597669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.598306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.599932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.600509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb7b7c-af93-4ff1-9fa8-e8518712a859 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.605516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.606161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.606640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.606923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] PATH_INFO: `/v3/projects/6e1d9770a05b4531907c076c9433d05a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.608124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1d9770a05b4531907c076c9433d05a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.610095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.612071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1160/2330] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:31.615937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.626151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e78afc-4421-46a3-83cd-b631d5858cd7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.630721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.631303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.631583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.631875 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.632519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.634164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.644697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1161/2331] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:31.649157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.659546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094905e-0cf8-4989-b576-4e967882b5d8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.663220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1d9770a05b4531907c076c9433d05a', 'request_id': 'req-8548955e-f878-468b-ad40-85e776333ca3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.668716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1d9770a05b4531907c076c9433d05a', 'request_id': 'req-8548955e-f878-468b-ad40-85e776333ca3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.675862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1d9770a05b4531907c076c9433d05a', 'request_id': 'req-8548955e-f878-468b-ad40-85e776333ca3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.692531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] The token cache is being invalidate because project 6e1d9770a05b4531907c076c9433d05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:31.696283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.697048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8548955e-f878-468b-ad40-85e776333ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:31.698659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1171/2332] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:31 2022] DELETE /identity/v3/projects/6e1d9770a05b4531907c076c9433d05a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:31.703460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdc087-6f5c-4a1a-9117-9b07740291fd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.703779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdc087-6f5c-4a1a-9117-9b07740291fd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.704098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdc087-6f5c-4a1a-9117-9b07740291fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.722018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792756bf-e753-4194-a6b8-775b26e60c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.722681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792756bf-e753-4194-a6b8-775b26e60c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.722961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792756bf-e753-4194-a6b8-775b26e60c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.723230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792756bf-e753-4194-a6b8-775b26e60c97 admin admin] PATH_INFO: `/v3/projects/74c29af7edf4443194001885da6b7efb/users/8ae1a0e09108418e89b373fcac3a1fcc/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.730484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-41bdc087-6f5c-4a1a-9117-9b07740291fd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:31.731813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-41bdc087-6f5c-4a1a-9117-9b07740291fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.738657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792756bf-e753-4194-a6b8-775b26e60c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c29af7edf4443194001885da6b7efb, user_id=8ae1a0e09108418e89b373fcac3a1fcc, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.743896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1172/2333] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:31.744054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792756bf-e753-4194-a6b8-775b26e60c97 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.749492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be7ea4-2ea0-4871-ad7c-be1847a2b576 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.750234 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be7ea4-2ea0-4871-ad7c-be1847a2b576 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.750683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be7ea4-2ea0-4871-ad7c-be1847a2b576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.756996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1162/2334] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:31 2022] PUT /identity/v3/projects/74c29af7edf4443194001885da6b7efb/users/8ae1a0e09108418e89b373fcac3a1fcc/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:31.766845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be048d1-ffa5-447a-b16e-c335ca636b04 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.767731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be048d1-ffa5-447a-b16e-c335ca636b04 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.768231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be048d1-ffa5-447a-b16e-c335ca636b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.773556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-79be7ea4-2ea0-4871-ad7c-be1847a2b576 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:31.774770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-79be7ea4-2ea0-4871-ad7c-be1847a2b576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.798330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2be048d1-ffa5-447a-b16e-c335ca636b04 None None] MFA Rules not processed for user `8ae1a0e09108418e89b373fcac3a1fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:31.799877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2be048d1-ffa5-447a-b16e-c335ca636b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.843674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1173/2335] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:31.849934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be7ea4-2ea0-4871-ad7c-be1847a2b576 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.864200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be7ea4-2ea0-4871-ad7c-be1847a2b576 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.870002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.871108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.871230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.871398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.872118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:31.872408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.874153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.887374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1163/2336] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:31.891067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1174/2337] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:06:31.893343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be048d1-ffa5-447a-b16e-c335ca636b04 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.899613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.901332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:31.908858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be048d1-ffa5-447a-b16e-c335ca636b04 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:31.910301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:31.914477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:31.915535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:31.915809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:31.916228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:31.917031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:31.919094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:31.956868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1164/2338] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:31.966266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:31.990513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820aae5-83d7-4cdb-b46f-560bade92c59 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:32.004940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:32.079606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:32.081400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:32.081697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:32.081996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:32.088452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6ee82-38bd-4596-80e4-e8758ad1d012 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:32.094821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:32.094821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:32.094821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:32.095106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:32.095699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:32.097649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:32.121104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1165/2339] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:31 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:32.124715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:32.137451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d72c4-83eb-4066-980b-fdb8a2f13dee admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:32.143766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:32.144402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:32.144684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:32.144962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:32.145600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:32.147478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:32.155693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:32.157197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:32.158835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1166/2340] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:32.162959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:32.166394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1175/2341] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:32.172844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436b3f2-04fe-4015-b64d-aeadf3cbaf81 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:32.177492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc22eef-8403-44f0-a369-19bee2c5563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:32.178079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22eef-8403-44f0-a369-19bee2c5563f admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:32.178350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22eef-8403-44f0-a369-19bee2c5563f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:32.179281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22eef-8403-44f0-a369-19bee2c5563f admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:32.179281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc22eef-8403-44f0-a369-19bee2c5563f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:32.180860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc22eef-8403-44f0-a369-19bee2c5563f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:32.191652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1167/2342] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:32.196003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:32.207862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01f85-43ff-4728-9ebc-a61f8d6cd1b7 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:32.212641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:32.213400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:32.213678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:32.213959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] PATH_INFO: `/v3/projects/fb718a8a6011481c8c425b25f426aa3f/users/bb58f4400de84c13ab7ffda8490f28cd/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:32.228866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb718a8a6011481c8c425b25f426aa3f, user_id=bb58f4400de84c13ab7ffda8490f28cd, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:32.231564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:32.244750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1176/2343] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:32 2022] PUT /identity/v3/projects/fb718a8a6011481c8c425b25f426aa3f/users/bb58f4400de84c13ab7ffda8490f28cd/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:32.249876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492ab3c-0e96-4675-ae73-3c9549dd02f3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:32.250266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492ab3c-0e96-4675-ae73-3c9549dd02f3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:32.250543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492ab3c-0e96-4675-ae73-3c9549dd02f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:32.271278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b492ab3c-0e96-4675-ae73-3c9549dd02f3 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:32.272857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b492ab3c-0e96-4675-ae73-3c9549dd02f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:32.335730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1168/2344] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:32.345409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:32.347415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:32.415069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:32.422320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eb9d9-2d92-4f1f-89cb-44f18d9dbd69 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:32.427687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:32.429147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:32.429147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:32.429495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:32.441835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:32.443673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:32.455184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1177/2345] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:33.094593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:33.095370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:33.095939 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:33.131391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] MFA Rules not processed for user `bb58f4400de84c13ab7ffda8490f28cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:33.136107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:33.253731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1169/2346] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:33.266569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:33.367993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4e85-eb2d-448f-bd76-936bcbfe4444 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:33.380375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1edf82-6ce7-4cf0-b4ef-28597e1b4f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:33.381009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1edf82-6ce7-4cf0-b4ef-28597e1b4f37 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:33.381375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1edf82-6ce7-4cf0-b4ef-28597e1b4f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:33.381554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1edf82-6ce7-4cf0-b4ef-28597e1b4f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:33.399966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1edf82-6ce7-4cf0-b4ef-28597e1b4f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:33.401425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1edf82-6ce7-4cf0-b4ef-28597e1b4f37 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:33.411796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1178/2347] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:34.894606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:34.896068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:34.910279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2fa2c-05b1-42dd-bdf2-754e1659cc48 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:34.914687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:34.915296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:34.915570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:34.915863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] PATH_INFO: `/v3/users/643fecf195ff46a7a5e6d4e35c438813` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:34.916955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] RBAC: Authorizing `identity:delete_user(user_id=643fecf195ff46a7a5e6d4e35c438813)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:34.918870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:34.923382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:34.967531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643fecf195ff46a7a5e6d4e35c438813'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:34.979827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643fecf195ff46a7a5e6d4e35c438813'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:34.980503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643fecf195ff46a7a5e6d4e35c438813'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:34.988504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643fecf195ff46a7a5e6d4e35c438813'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:34.989110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643fecf195ff46a7a5e6d4e35c438813'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.001249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1170/2348] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:34 2022] DELETE /identity/v3/users/643fecf195ff46a7a5e6d4e35c438813 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:35.005728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:35.006244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:35.006341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:35.025798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:35.026942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:35.087660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1179/2349] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:35.097134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:35.098543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:35.156171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d697b5f-557b-4b7f-97a2-e718deb1f89c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:35.160108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:35.160700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:35.160978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:35.161250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:35.170097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:35.171520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:35.181150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1171/2350] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:35.454940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:35.456482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:35.469528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1963f4-49b8-46ae-a011-b9f4024c26af None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:35.474808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:35.475482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:35.475766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:35.476198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] PATH_INFO: `/v3/projects/6861ba7333684225b0e72933031463cd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:35.477196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6861ba7333684225b0e72933031463cd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:35.479247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:35.528569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6861ba7333684225b0e72933031463cd', 'request_id': 'req-8fc28c1f-1052-44fa-9e7e-abe2179d7110'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.534313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6861ba7333684225b0e72933031463cd', 'request_id': 'req-8fc28c1f-1052-44fa-9e7e-abe2179d7110'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.540737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6861ba7333684225b0e72933031463cd', 'request_id': 'req-8fc28c1f-1052-44fa-9e7e-abe2179d7110'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.553952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] The token cache is being invalidate because project 6861ba7333684225b0e72933031463cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:35.554246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.554881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.556543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1180/2351] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:35 2022] DELETE /identity/v3/projects/6861ba7333684225b0e72933031463cd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:35.560692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:35.571298 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b972-030b-41d0-83ea-fae90f12297e service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:35.630431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:35.631063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:35.631343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:35.631616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] PATH_INFO: `/v3/users/ae9996e3871c40a188c4486ca9776764` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:35.632760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9996e3871c40a188c4486ca9776764)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:35.634760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:35.663856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:35.683669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9996e3871c40a188c4486ca9776764'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.696380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9996e3871c40a188c4486ca9776764'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.697055 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9996e3871c40a188c4486ca9776764'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.704342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9996e3871c40a188c4486ca9776764'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.705325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9996e3871c40a188c4486ca9776764'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:35.717714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1172/2352] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:35 2022] DELETE /identity/v3/users/ae9996e3871c40a188c4486ca9776764 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:35.991771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:35.993398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:36.006903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28c1f-1052-44fa-9e7e-abe2179d7110 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:36.021461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:36.066483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:36.067028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:36.067308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:36.067581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] PATH_INFO: `/v3/projects/f10c2d3561ec40e6b6199c25c33e3273` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:36.068718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10c2d3561ec40e6b6199c25c33e3273)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:36.070609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:36.116465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10c2d3561ec40e6b6199c25c33e3273', 'request_id': 'req-51af4e9c-615e-4793-8236-c1b06fe94f86'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.121821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10c2d3561ec40e6b6199c25c33e3273', 'request_id': 'req-51af4e9c-615e-4793-8236-c1b06fe94f86'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.129202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10c2d3561ec40e6b6199c25c33e3273', 'request_id': 'req-51af4e9c-615e-4793-8236-c1b06fe94f86'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.140390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] The token cache is being invalidate because project f10c2d3561ec40e6b6199c25c33e3273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:36.140681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.141310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.143425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1181/2353] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:35 2022] DELETE /identity/v3/projects/f10c2d3561ec40e6b6199c25c33e3273 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:36.147761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:36.149173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:36.161742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a93e9-aa07-4660-b151-a4cd369d34d7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:36.217747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:36.218369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:36.218650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:36.218933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] PATH_INFO: `/v3/users/f02d1b4a67854d65a9135e1fc126759a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:36.220137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02d1b4a67854d65a9135e1fc126759a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:36.222503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:36.268525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02d1b4a67854d65a9135e1fc126759a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.280459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02d1b4a67854d65a9135e1fc126759a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.281114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02d1b4a67854d65a9135e1fc126759a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.288178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02d1b4a67854d65a9135e1fc126759a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.288780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c43880cd-74c3-4914-9a2c-0974f04467bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02d1b4a67854d65a9135e1fc126759a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.301213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1173/2354] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:36 2022] DELETE /identity/v3/users/f02d1b4a67854d65a9135e1fc126759a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:36.570937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:36.572394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:36.586194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af4e9c-615e-4793-8236-c1b06fe94f86 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:36.644381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:36.645008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:36.645290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:36.645564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] PATH_INFO: `/v3/projects/2458b6a3de3146d1979ca09397709b99` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:36.646778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2458b6a3de3146d1979ca09397709b99)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:36.649003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:36.699934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2458b6a3de3146d1979ca09397709b99', 'request_id': 'req-f52dcbae-5a19-4000-9370-a0b1314ba71d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.701231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:36.707248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2458b6a3de3146d1979ca09397709b99', 'request_id': 'req-f52dcbae-5a19-4000-9370-a0b1314ba71d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.714322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2458b6a3de3146d1979ca09397709b99', 'request_id': 'req-f52dcbae-5a19-4000-9370-a0b1314ba71d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.727784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] The token cache is being invalidate because project 2458b6a3de3146d1979ca09397709b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:36.728163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.729355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f52dcbae-5a19-4000-9370-a0b1314ba71d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:36.730842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1182/2355] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:36 2022] DELETE /identity/v3/projects/2458b6a3de3146d1979ca09397709b99 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:36.735701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76380961-1221-4cf8-8c74-06f7ce80757b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:36.736667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76380961-1221-4cf8-8c74-06f7ce80757b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:36.736942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76380961-1221-4cf8-8c74-06f7ce80757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:36.756603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-76380961-1221-4cf8-8c74-06f7ce80757b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:36.757800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-76380961-1221-4cf8-8c74-06f7ce80757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:36.776690 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:36.821418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1174/2356] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:36.828972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8aa391-72de-43f9-899a-1a8b8fdd8110 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:36.829562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8aa391-72de-43f9-899a-1a8b8fdd8110 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:36.830044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8aa391-72de-43f9-899a-1a8b8fdd8110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:36.854551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9e8aa391-72de-43f9-899a-1a8b8fdd8110 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:36.855729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9e8aa391-72de-43f9-899a-1a8b8fdd8110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:36.864409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1183/2357] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:36.868721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76380961-1221-4cf8-8c74-06f7ce80757b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:36.878577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76380961-1221-4cf8-8c74-06f7ce80757b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:36.883149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:36.883744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:36.884060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:36.884337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:36.884987 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:36.885291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:36.886883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:36.898711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1175/2358] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:06:36.902904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8aa391-72de-43f9-899a-1a8b8fdd8110 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:36.912729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8aa391-72de-43f9-899a-1a8b8fdd8110 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:36.917472 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:36.918071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:36.918350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:36.918624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:36.919266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:36.921215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:36.949401 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1184/2359] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:36 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:36.953615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:36.964167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc34be-292f-4b0e-9313-0e353db0e73d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.020306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.020907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.021208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.021480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.022359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.024287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.044794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1176/2360] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:36 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:37.049276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.059577 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db071cfa-b49d-43a7-93ee-8fc5384630fd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.064731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.065026 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.065403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.065757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.066532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.068221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.081473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1185/2361] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:37.085581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.096006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bc92c-64c8-46be-b94a-ddc645c902f4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.100656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059728ae-5281-452e-a6bc-368a0b734152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.101252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059728ae-5281-452e-a6bc-368a0b734152 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.101535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059728ae-5281-452e-a6bc-368a0b734152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.101810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059728ae-5281-452e-a6bc-368a0b734152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.102452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059728ae-5281-452e-a6bc-368a0b734152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.104077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059728ae-5281-452e-a6bc-368a0b734152 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.115134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1177/2362] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:37.119719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.129962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9112d0-1ff9-4331-8813-70b09e574268 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.134592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.135240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.135523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.135801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] PATH_INFO: `/v3/projects/8fd92538ee894a22bc5dc41861014c0a/users/338c212570714a16a181f48c1ef46893/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.150740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd92538ee894a22bc5dc41861014c0a, user_id=338c212570714a16a181f48c1ef46893, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.153431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.167102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1186/2363] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:37 2022] PUT /identity/v3/projects/8fd92538ee894a22bc5dc41861014c0a/users/338c212570714a16a181f48c1ef46893/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:37.177576 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88397a4a-5f30-47e2-bc9d-21c72199509a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.178040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88397a4a-5f30-47e2-bc9d-21c72199509a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.178214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88397a4a-5f30-47e2-bc9d-21c72199509a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.205684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-88397a4a-5f30-47e2-bc9d-21c72199509a None None] MFA Rules not processed for user `338c212570714a16a181f48c1ef46893`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:37.206890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-88397a4a-5f30-47e2-bc9d-21c72199509a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:37.269677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1178/2364] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:37.274481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.336209 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f381fc-b867-4081-a638-fceaf3940e78 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.340840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.341462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.341748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.342025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.342654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.344592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.370734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1187/2365] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:37 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:37.374927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88397a4a-5f30-47e2-bc9d-21c72199509a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.386036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88397a4a-5f30-47e2-bc9d-21c72199509a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.403881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:37.445455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.446050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.446329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.446603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.447228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.449163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.469550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1179/2366] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:37 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:06:37.474248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.484475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f737550-364a-4a82-9d28-9a81ade80cca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.490046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.490642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.490964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.491227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.491900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.493493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.504018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1188/2367] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:37.507785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.517629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ef2d9-2b1e-4a3a-8a52-7bc40736776e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.522465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f8489-a38c-45bf-a1ff-5d6419a9c059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.523047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f8489-a38c-45bf-a1ff-5d6419a9c059 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.523326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f8489-a38c-45bf-a1ff-5d6419a9c059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.523594 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f8489-a38c-45bf-a1ff-5d6419a9c059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.524266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f8489-a38c-45bf-a1ff-5d6419a9c059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.525829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f8489-a38c-45bf-a1ff-5d6419a9c059 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.536090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1180/2368] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:37.540150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.551075 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258d0c8-909a-4547-a20b-65eeedd6d443 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.555828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.556494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.556770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.557041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] PATH_INFO: `/v3/projects/b08baa03fd4d44c68e7560ba9d20b961/users/9c5c9b6402bd4b788fb1915d08c2e7d2/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.571238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08baa03fd4d44c68e7560ba9d20b961, user_id=9c5c9b6402bd4b788fb1915d08c2e7d2, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.573891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.586867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1189/2369] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:37 2022] PUT /identity/v3/projects/b08baa03fd4d44c68e7560ba9d20b961/users/9c5c9b6402bd4b788fb1915d08c2e7d2/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:37.598308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596097-b183-40da-ad51-f74b58acf8a3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.598621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596097-b183-40da-ad51-f74b58acf8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.598899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596097-b183-40da-ad51-f74b58acf8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.627687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a5596097-b183-40da-ad51-f74b58acf8a3 None None] MFA Rules not processed for user `9c5c9b6402bd4b788fb1915d08c2e7d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:37.628924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a5596097-b183-40da-ad51-f74b58acf8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:37.692726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1181/2370] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:37.697174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.725880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:37.761676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4260beb-c27c-466c-9d4b-2b15b477adf0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.766496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.767732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.768052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.768323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.769086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.770976 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.797601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1190/2371] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:37 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:37.801697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5596097-b183-40da-ad51-f74b58acf8a3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.812150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5596097-b183-40da-ad51-f74b58acf8a3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.873300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.874215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.874559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.874897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.875605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.877860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.905193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1182/2372] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:37 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:37.907367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.918872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54877b28-7815-47fd-8b0f-7127ac9d84d4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.923990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.924531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.924879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.925259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.926033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.927727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.938597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1191/2373] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:37.943034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:37.953607 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f46a12-5006-4c54-83d0-01f2410de42f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:37.958964 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:37.959678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:37.960050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:37.960405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] PATH_INFO: `/v3/projects/f6ee8aacfa284b58b95d61e1b88f03f8/users/d22698e882904b0d8f33bbb2760b60df/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:37.977976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ee8aacfa284b58b95d61e1b88f03f8, user_id=d22698e882904b0d8f33bbb2760b60df, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:37.980827 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:37.983296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:37.998228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1183/2374] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:37 2022] PUT /identity/v3/projects/f6ee8aacfa284b58b95d61e1b88f03f8/users/d22698e882904b0d8f33bbb2760b60df/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:38.003171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:38.015481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d096afc-ec25-46d2-890b-da50e15343b6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:38.078569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912d691-3282-4383-b98d-51676f0e5edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:38.079247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912d691-3282-4383-b98d-51676f0e5edc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:38.080081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912d691-3282-4383-b98d-51676f0e5edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:38.080375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912d691-3282-4383-b98d-51676f0e5edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:38.081020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912d691-3282-4383-b98d-51676f0e5edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:38.082612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912d691-3282-4383-b98d-51676f0e5edc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:38.092391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1192/2375] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:06:38.097384 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:38.109967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab2cbe-4842-448b-8483-49defd0b8e63 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:38.114989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:38.115586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:38.115907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:38.116183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] PATH_INFO: `/v3/projects/f6ee8aacfa284b58b95d61e1b88f03f8/users/d22698e882904b0d8f33bbb2760b60df/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:38.118110 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ee8aacfa284b58b95d61e1b88f03f8, user_id=d22698e882904b0d8f33bbb2760b60df, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:38.120855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:38.132175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1184/2376] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:38 2022] PUT /identity/v3/projects/f6ee8aacfa284b58b95d61e1b88f03f8/users/d22698e882904b0d8f33bbb2760b60df/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:38.142800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc176c-af13-4d24-8cd8-b68d6088b7af None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:38.143089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc176c-af13-4d24-8cd8-b68d6088b7af None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:38.143362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc176c-af13-4d24-8cd8-b68d6088b7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:38.170722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-78dc176c-af13-4d24-8cd8-b68d6088b7af None None] MFA Rules not processed for user `d22698e882904b0d8f33bbb2760b60df`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:38.171958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78dc176c-af13-4d24-8cd8-b68d6088b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:38.237039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1193/2377] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:38.248173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:38.249950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:38.308265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f76d1-3a41-4d36-a599-d2354f00af4c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:38.311994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:38.312584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:38.312860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:38.313128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:38.369769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:38.371137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:38.380541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1185/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:40.162713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c85a4e-5507-498c-ac84-0278c468a2a2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:40.163237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c85a4e-5507-498c-ac84-0278c468a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:40.163336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c85a4e-5507-498c-ac84-0278c468a2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:40.182417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-96c85a4e-5507-498c-ac84-0278c468a2a2 None None] MFA Rules not processed for user `bb58f4400de84c13ab7ffda8490f28cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:40.183604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-96c85a4e-5507-498c-ac84-0278c468a2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:40.196338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:40.238177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1194/2379] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:47.271723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:47.273492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:47.284737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0e123-7188-4ed5-8c5f-c105c187b0bd service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:47.288324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:47.288950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:47.289231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:47.289507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:47.345139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:47.346531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:47.355095 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1186/2380] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:49.950613 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c85a4e-5507-498c-ac84-0278c468a2a2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:49.960709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c85a4e-5507-498c-ac84-0278c468a2a2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:49.964201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00982ebc-8a42-4343-b96d-0a2e1f57a067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:49.964819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00982ebc-8a42-4343-b96d-0a2e1f57a067 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:49.965082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00982ebc-8a42-4343-b96d-0a2e1f57a067 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:49.965357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00982ebc-8a42-4343-b96d-0a2e1f57a067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:49.974811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00982ebc-8a42-4343-b96d-0a2e1f57a067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:49.976275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00982ebc-8a42-4343-b96d-0a2e1f57a067 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:49.985611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1195/2381] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:56.024277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:56.059386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:56.092335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65043b9-ce6c-4beb-897c-b22896b0d6c2 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:56.097842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:56.098408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:56.098696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:56.098975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] PATH_INFO: `/v3/users/338c212570714a16a181f48c1ef46893` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:56.100128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=338c212570714a16a181f48c1ef46893)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:56.102054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:56.151817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338c212570714a16a181f48c1ef46893'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.165486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338c212570714a16a181f48c1ef46893'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.166200 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338c212570714a16a181f48c1ef46893'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.174791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338c212570714a16a181f48c1ef46893'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.175425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338c212570714a16a181f48c1ef46893'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.189578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1187/2382] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:56 2022] DELETE /identity/v3/users/338c212570714a16a181f48c1ef46893 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:56.194811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844dd97-6e18-4cdf-a737-bdf957b2b944 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:56.195328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844dd97-6e18-4cdf-a737-bdf957b2b944 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:56.195426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844dd97-6e18-4cdf-a737-bdf957b2b944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:56.215525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4844dd97-6e18-4cdf-a737-bdf957b2b944 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:56.216798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4844dd97-6e18-4cdf-a737-bdf957b2b944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:56.280910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1196/2383] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:56.291976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:56.293545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:56.357108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c4406-5a8a-4e33-86c4-bbb3e4a36986 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:56.360925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:56.361621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:56.362586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:56.362869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:56.372212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:56.373736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:56.383443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1188/2384] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:06:56.638061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54811c-67c7-4ce5-92ca-f80cd460db25 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:56.638354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54811c-67c7-4ce5-92ca-f80cd460db25 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:56.638633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54811c-67c7-4ce5-92ca-f80cd460db25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:56.663863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9b54811c-67c7-4ce5-92ca-f80cd460db25 None None] MFA Rules not processed for user `f1815cf2b3f347eb8276769cc30cf116`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:56.665145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b54811c-67c7-4ce5-92ca-f80cd460db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:56.692380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:56.692380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:56.696901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0cf3-1a78-428c-bd4f-3f05e54933a8 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:56.706611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:56.707258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:56.707536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:56.707870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] PATH_INFO: `/v3/projects/8fd92538ee894a22bc5dc41861014c0a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:56.708988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd92538ee894a22bc5dc41861014c0a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:56.711728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:56.751661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1197/2385] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:56.764305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd92538ee894a22bc5dc41861014c0a', 'request_id': 'req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.765482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:56.772495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd92538ee894a22bc5dc41861014c0a', 'request_id': 'req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.780415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd92538ee894a22bc5dc41861014c0a', 'request_id': 'req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.794663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] The token cache is being invalidate because project 8fd92538ee894a22bc5dc41861014c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:56.794957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.795648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.797623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1189/2386] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:56 2022] DELETE /identity/v3/projects/8fd92538ee894a22bc5dc41861014c0a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:56.801620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54811c-67c7-4ce5-92ca-f80cd460db25 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:56.812616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54811c-67c7-4ce5-92ca-f80cd460db25 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:56.865871 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:56.872874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:56.873520 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:56.873803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:56.874079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] PATH_INFO: `/v3/users/9c5c9b6402bd4b788fb1915d08c2e7d2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:56.875217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c5c9b6402bd4b788fb1915d08c2e7d2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:56.878032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:56.937561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c5c9b6402bd4b788fb1915d08c2e7d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.953279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c5c9b6402bd4b788fb1915d08c2e7d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.966444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c5c9b6402bd4b788fb1915d08c2e7d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.966562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c5c9b6402bd4b788fb1915d08c2e7d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.975454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c5c9b6402bd4b788fb1915d08c2e7d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:56.984439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1198/2387] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:56 2022] DELETE /identity/v3/users/9c5c9b6402bd4b788fb1915d08c2e7d2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:57.431856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:57.433655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:57.453206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5753dcc-58cc-4983-81ec-3a8bfd40bc1e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:57.524088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:57.524830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:57.525179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:57.525526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] PATH_INFO: `/v3/projects/b08baa03fd4d44c68e7560ba9d20b961` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:57.526728 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08baa03fd4d44c68e7560ba9d20b961)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:57.528857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:57.592215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08baa03fd4d44c68e7560ba9d20b961', 'request_id': 'req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.599565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08baa03fd4d44c68e7560ba9d20b961', 'request_id': 'req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.606679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08baa03fd4d44c68e7560ba9d20b961', 'request_id': 'req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.621748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] The token cache is being invalidate because project b08baa03fd4d44c68e7560ba9d20b961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:57.622120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.622762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.624343 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1190/2388] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:57 2022] DELETE /identity/v3/projects/b08baa03fd4d44c68e7560ba9d20b961 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:57.628470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:57.630025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:57.642918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf5c46-3b29-42ad-a33a-692db5e8a7fa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:57.703863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:57.704517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:57.704798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:57.705070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] PATH_INFO: `/v3/users/d22698e882904b0d8f33bbb2760b60df` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:57.706397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22698e882904b0d8f33bbb2760b60df)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:57.708895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:57.763001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22698e882904b0d8f33bbb2760b60df'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.777503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22698e882904b0d8f33bbb2760b60df'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.787414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22698e882904b0d8f33bbb2760b60df'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.788228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22698e882904b0d8f33bbb2760b60df'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.794558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:57.796879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22698e882904b0d8f33bbb2760b60df'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:57.806527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1199/2389] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:57 2022] DELETE /identity/v3/users/d22698e882904b0d8f33bbb2760b60df => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:58.053450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.054896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:58.067759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb6c5-0c4f-4f16-96a7-23df18e22825 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.109957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.111479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:58.137743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.139398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.139895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.140327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] PATH_INFO: `/v3/users/8ae1a0e09108418e89b373fcac3a1fcc` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.141866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae1a0e09108418e89b373fcac3a1fcc)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.145367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.161591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:58.217703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cdac4-1566-4306-bebf-16ddee03547f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.222145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae1a0e09108418e89b373fcac3a1fcc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.223393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.224313 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.224536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.224816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] PATH_INFO: `/v3/projects/f6ee8aacfa284b58b95d61e1b88f03f8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.226445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ee8aacfa284b58b95d61e1b88f03f8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.228754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.240024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae1a0e09108418e89b373fcac3a1fcc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.241194 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae1a0e09108418e89b373fcac3a1fcc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.250462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae1a0e09108418e89b373fcac3a1fcc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.256180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6ba78f39-e52e-447f-bb59-f89fe417ca7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae1a0e09108418e89b373fcac3a1fcc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.280309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1191/2390] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:06:58 2022] DELETE /identity/v3/users/8ae1a0e09108418e89b373fcac3a1fcc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:58.285858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.286704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.287288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.294639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ee8aacfa284b58b95d61e1b88f03f8', 'request_id': 'req-d341e517-62df-4ea6-b2a4-475f493874e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.301766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ee8aacfa284b58b95d61e1b88f03f8', 'request_id': 'req-d341e517-62df-4ea6-b2a4-475f493874e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.312296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:58.313689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ee8aacfa284b58b95d61e1b88f03f8', 'request_id': 'req-d341e517-62df-4ea6-b2a4-475f493874e5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.314105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:58.332937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] The token cache is being invalidate because project f6ee8aacfa284b58b95d61e1b88f03f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:58.332937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.333197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d341e517-62df-4ea6-b2a4-475f493874e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:58.335282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1200/2391] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:58 2022] DELETE /identity/v3/projects/f6ee8aacfa284b58b95d61e1b88f03f8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:58.339804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb317-8536-467a-b64f-e03bd60b0c8c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.340158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb317-8536-467a-b64f-e03bd60b0c8c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.340492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb317-8536-467a-b64f-e03bd60b0c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.361115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb5cb317-8536-467a-b64f-e03bd60b0c8c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:58.362940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb5cb317-8536-467a-b64f-e03bd60b0c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:58.391678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1192/2392] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:58.401966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.403533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:58.460564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1201/2393] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:58.468277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205baf-10d9-4fdf-a1fc-d5b4cd89db76 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.468555 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205baf-10d9-4fdf-a1fc-d5b4cd89db76 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.468828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205baf-10d9-4fdf-a1fc-d5b4cd89db76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.479326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f291d-2c6f-4f82-958d-84c059576be7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.484034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.484855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.485133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.485397 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.495402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d9205baf-10d9-4fdf-a1fc-d5b4cd89db76 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:58.497040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d9205baf-10d9-4fdf-a1fc-d5b4cd89db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:58.504856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1202/2394] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:06:58.505962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.507703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.509007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205baf-10d9-4fdf-a1fc-d5b4cd89db76 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.519516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1193/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:06:58.522358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205baf-10d9-4fdf-a1fc-d5b4cd89db76 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.527786 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.528961 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.529245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.529572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.530228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:06:58.530528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.532181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.548029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1203/2396] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:06:58.552075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.563973 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be37e5-5a49-4979-a1c5-cd459fdb00d9 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.579669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.583018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.583571 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.583867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.585711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.589228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.671889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1194/2397] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:06:58 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:58.674749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.685813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dfe98-b918-4d50-885a-186bed39facb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.690966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:58.773599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.774532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.774532 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.774707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.775440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.777482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.800181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1204/2398] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:06:58 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:06:58.804339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.814627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d256bea-62c2-47c0-8b45-4fc4b0ba5f50 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.820289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.820918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.821204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.821477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.822121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.823723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.826028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:58.836725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1195/2399] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:58.840167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.851437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507e0d-b78c-43b7-8059-10d963a58a09 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.856227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.856862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.857198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.857474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.858148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.859752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.875115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1205/2400] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:06:58.876325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.886575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8322e-0142-445c-87fa-cc6ad6296380 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:58.891261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b8507-fba1-4ed6-a3b9-f0291f0fa46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:58.891948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b8507-fba1-4ed6-a3b9-f0291f0fa46d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:58.892232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b8507-fba1-4ed6-a3b9-f0291f0fa46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:58.892509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b8507-fba1-4ed6-a3b9-f0291f0fa46d admin admin] PATH_INFO: `/v3/projects/dc88ab1bf75e47e2bdd03d3cd3aa0eee/users/aa74dc9683a3492b9a71c36b5346c686/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:58.910913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b8507-fba1-4ed6-a3b9-f0291f0fa46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc88ab1bf75e47e2bdd03d3cd3aa0eee, user_id=aa74dc9683a3492b9a71c36b5346c686, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:58.913689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b8507-fba1-4ed6-a3b9-f0291f0fa46d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:58.929491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1196/2401] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:06:58 2022] PUT /identity/v3/projects/dc88ab1bf75e47e2bdd03d3cd3aa0eee/users/aa74dc9683a3492b9a71c36b5346c686/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:06:58.966711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:58.968120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:59.005826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:59.006159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:59.006372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:59.009855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b9afe-d1ea-45b6-a378-64116a93278b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:59.046063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] MFA Rules not processed for user `aa74dc9683a3492b9a71c36b5346c686`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:06:59.047359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:59.133673 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1197/2402] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:06:59.135683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:59.136353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:59.136632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:59.136910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] PATH_INFO: `/v3/projects/74c29af7edf4443194001885da6b7efb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:59.138109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c29af7edf4443194001885da6b7efb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:59.140120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:59.146164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:06:59.147792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:59.203111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c29af7edf4443194001885da6b7efb', 'request_id': 'req-703dac66-3787-4f80-84b6-a033be6bf057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:59.210981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c29af7edf4443194001885da6b7efb', 'request_id': 'req-703dac66-3787-4f80-84b6-a033be6bf057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:59.217057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769982c-6c4d-4588-bf62-2f7808448d00 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:06:59.219014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c29af7edf4443194001885da6b7efb', 'request_id': 'req-703dac66-3787-4f80-84b6-a033be6bf057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:59.239637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] The token cache is being invalidate because project 74c29af7edf4443194001885da6b7efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:06:59.239873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:59.240795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-703dac66-3787-4f80-84b6-a033be6bf057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:06:59.245791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1206/2403] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:06:58 2022] DELETE /identity/v3/projects/74c29af7edf4443194001885da6b7efb => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:06:59.294792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:06:59.295423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:06:59.295701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:06:59.296019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:06:59.298356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:06:59.341455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:06:59.377696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:06:59.379320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:06:59.390620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1198/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:06:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:07:00.775812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce91e-dd29-4a6e-ba95-7b1d667af20c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:00.776366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce91e-dd29-4a6e-ba95-7b1d667af20c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:00.776366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce91e-dd29-4a6e-ba95-7b1d667af20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:00.796443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-896ce91e-dd29-4a6e-ba95-7b1d667af20c None None] MFA Rules not processed for user `aa74dc9683a3492b9a71c36b5346c686`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:00.797637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-896ce91e-dd29-4a6e-ba95-7b1d667af20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:00.804814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1207/2405] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:09.385431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=12 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:10.791390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:10.909514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:10.911072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:10.999330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a7fb-e536-4733-bfc1-6caa3c6a833b service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:11.005236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9202b-81f1-4295-9b78-73c9dd747625 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:11.005850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9202b-81f1-4295-9b78-73c9dd747625 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:11.006192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9202b-81f1-4295-9b78-73c9dd747625 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:11.006473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9202b-81f1-4295-9b78-73c9dd747625 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:11.008303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e5f9202b-81f1-4295-9b78-73c9dd747625 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:11.022533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9202b-81f1-4295-9b78-73c9dd747625 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:11.024174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9202b-81f1-4295-9b78-73c9dd747625 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:11.035062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1199/2406] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:07:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:07:17.261960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:17.262429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:17.262521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:17.282745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] MFA Rules not processed for user `bb58f4400de84c13ab7ffda8490f28cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:17.285094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:17.339768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1208/2407] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:07:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:21.030471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=32 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:27.333227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=25 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:30.334562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade62f4-68d7-4298-adf9-f4ce13d94dba None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:30.335065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade62f4-68d7-4298-adf9-f4ce13d94dba None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:30.335171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade62f4-68d7-4298-adf9-f4ce13d94dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:30.355825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-eade62f4-68d7-4298-adf9-f4ce13d94dba None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:30.357042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eade62f4-68d7-4298-adf9-f4ce13d94dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:30.412088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1200/2408] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:07:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:30.451995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:30.453403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:30.522983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675f607-cb7f-4ff8-b810-0908c30a5d09 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:30.527857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:30.528465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:30.528742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:30.529012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:30.537120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:30.538707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:30.546583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1209/2409] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:07:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:07:31.103781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f0b66-2443-4855-8ef9-9a37ed326fcd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:31.104096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f0b66-2443-4855-8ef9-9a37ed326fcd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:31.104370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f0b66-2443-4855-8ef9-9a37ed326fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:31.124345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-656f0b66-2443-4855-8ef9-9a37ed326fcd None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:31.125557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-656f0b66-2443-4855-8ef9-9a37ed326fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:31.133514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1201/2410] 10.222.0.52 () {58 vars in 967 bytes} [Thu Oct 6 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:31.147526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:31.170620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da95497-cbe9-4dfe-90f4-7a0361647e6c service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:31.179181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:31.179797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:31.180112 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:31.180387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:31.188899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:31.190684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:31.198703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1210/2411] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:07:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 06 11:07:31.467516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083f849-6c91-4dce-b20c-49908df38efc None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:31.468022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083f849-6c91-4dce-b20c-49908df38efc None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:31.468127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083f849-6c91-4dce-b20c-49908df38efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:31.487563 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7083f849-6c91-4dce-b20c-49908df38efc None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:31.488795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7083f849-6c91-4dce-b20c-49908df38efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:31.495820 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1202/2412] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:31.508404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:31.519589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2beb0d-8ed3-4e25-be6c-2dedf6f613f3 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:31.524831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:31.525417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:31.525695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:31.525964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:31.533915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:31.535473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:31.543318 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1211/2413] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:07:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:07:31.875556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:31.875859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:31.876155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:31.896028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:31.897460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:31.904548 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1203/2414] 10.222.0.52 () {58 vars in 967 bytes} [Thu Oct 6 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:31.916966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:31.928913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e97b2-3ac7-4a27-b3f1-8fcc64885278 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:31.934079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6402-48b8-4acd-9487-0bba6e71c193 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:31.934777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6402-48b8-4acd-9487-0bba6e71c193 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:31.935243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6402-48b8-4acd-9487-0bba6e71c193 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:31.935592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6402-48b8-4acd-9487-0bba6e71c193 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:31.944674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec6402-48b8-4acd-9487-0bba6e71c193 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:31.946443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec6402-48b8-4acd-9487-0bba6e71c193 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:31.954744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1212/2415] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:07:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:07:34.072136 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:34.073646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:34.140547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6206a-e1a6-4f5c-8fea-a850eec9ad2a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:34.145480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:34.146101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:34.146382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:34.146657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] PATH_INFO: `/v3/users/aa74dc9683a3492b9a71c36b5346c686` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:34.148096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa74dc9683a3492b9a71c36b5346c686)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:34.149999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:34.194734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa74dc9683a3492b9a71c36b5346c686'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.207334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa74dc9683a3492b9a71c36b5346c686'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.208026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa74dc9683a3492b9a71c36b5346c686'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.214944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa74dc9683a3492b9a71c36b5346c686'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.215547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa74dc9683a3492b9a71c36b5346c686'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.228219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1204/2416] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:07:34 2022] DELETE /identity/v3/users/aa74dc9683a3492b9a71c36b5346c686 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:07:34.232617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:34.233127 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:34.233224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:34.252295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:34.253585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:34.312671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1213/2417] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:34.322966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:34.324409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:34.331706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:34.385526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160487bc-1058-418e-bb43-6d54cf346115 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:34.389267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e586-a2f3-49a9-aec8-9c1c1e8fa62a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:34.389867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e586-a2f3-49a9-aec8-9c1c1e8fa62a service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:34.390149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e586-a2f3-49a9-aec8-9c1c1e8fa62a service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:34.390420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e586-a2f3-49a9-aec8-9c1c1e8fa62a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:34.400753 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e586-a2f3-49a9-aec8-9c1c1e8fa62a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:34.402189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e586-a2f3-49a9-aec8-9c1c1e8fa62a service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:34.411726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1205/2418] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:07:34.577166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:34.578609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:34.593215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1984166-f225-44d8-abca-4c6138dbfd4f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:34.598256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:34.599104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:34.599389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:34.599812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] PATH_INFO: `/v3/projects/dc88ab1bf75e47e2bdd03d3cd3aa0eee` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:34.601143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc88ab1bf75e47e2bdd03d3cd3aa0eee)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:34.602994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:34.651659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:34.655204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc88ab1bf75e47e2bdd03d3cd3aa0eee', 'request_id': 'req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.661643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc88ab1bf75e47e2bdd03d3cd3aa0eee', 'request_id': 'req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.667769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc88ab1bf75e47e2bdd03d3cd3aa0eee', 'request_id': 'req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.680383 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] The token cache is being invalidate because project dc88ab1bf75e47e2bdd03d3cd3aa0eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:07:34.680677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.681302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:07:34.682773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1214/2419] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:07:34 2022] DELETE /identity/v3/projects/dc88ab1bf75e47e2bdd03d3cd3aa0eee => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:07:42.973773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141b1dd-ad01-42e6-935e-f1b775971865 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:42.974330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141b1dd-ad01-42e6-935e-f1b775971865 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:42.974330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141b1dd-ad01-42e6-935e-f1b775971865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:42.993515 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d141b1dd-ad01-42e6-935e-f1b775971865 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:42.994689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d141b1dd-ad01-42e6-935e-f1b775971865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:43.049657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1206/2420] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:07:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:43.061776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:43.063165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:43.132360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b100a-c8dc-4e2f-aebb-a5a7cb40c5c3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:43.137531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:43.138134 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:43.138416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:43.138694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:43.153186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:43.154790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:43.164376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1215/2421] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:07:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:07:53.043088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=49 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:53.160576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:07:58.643524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:58.644128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:58.644128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:58.666997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:07:58.668171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:07:58.676182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1207/2422] 10.222.0.52 () {58 vars in 967 bytes} [Thu Oct 6 11:07:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:07:58.690403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:07:58.701222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eff3a-8aa7-4d9a-a488-d3cc31ad98cc service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:07:58.706682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:07:58.707281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:07:58.707556 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:07:58.707825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:07:58.716052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:07:58.717626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:07:58.726435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1216/2423] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:07:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:08:00.638935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:00.640454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:00.712770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465313e1-4a12-452b-9cf7-d519607e0b2d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:00.718223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:00.719169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:00.719440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:00.719720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] PATH_INFO: `/v3/users/bb58f4400de84c13ab7ffda8490f28cd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:00.721700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb58f4400de84c13ab7ffda8490f28cd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:00.723864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:00.784888 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb58f4400de84c13ab7ffda8490f28cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:00.797762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb58f4400de84c13ab7ffda8490f28cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:00.798557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb58f4400de84c13ab7ffda8490f28cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:00.807126 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb58f4400de84c13ab7ffda8490f28cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:00.807775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a410b42e-5d43-440d-bc7f-5a7f8bd42248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb58f4400de84c13ab7ffda8490f28cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:00.822118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1208/2424] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:08:00 2022] DELETE /identity/v3/users/bb58f4400de84c13ab7ffda8490f28cd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:08:01.047578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.058410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e21aed-f3ab-4a1a-9cd9-6c390c49ae4f service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.121877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.122516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.122803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.123081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] PATH_INFO: `/v3/projects/fb718a8a6011481c8c425b25f426aa3f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.124368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb718a8a6011481c8c425b25f426aa3f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.126278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.174311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb718a8a6011481c8c425b25f426aa3f', 'request_id': 'req-07043048-e15e-444b-9831-f91bc6516bcc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:01.181785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb718a8a6011481c8c425b25f426aa3f', 'request_id': 'req-07043048-e15e-444b-9831-f91bc6516bcc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:01.188351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb718a8a6011481c8c425b25f426aa3f', 'request_id': 'req-07043048-e15e-444b-9831-f91bc6516bcc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:01.204053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] The token cache is being invalidate because project fb718a8a6011481c8c425b25f426aa3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:08:01.204345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:01.205531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07043048-e15e-444b-9831-f91bc6516bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:08:01.207136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1217/2425] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:08:01 2022] DELETE /identity/v3/projects/fb718a8a6011481c8c425b25f426aa3f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:08:01.213096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edcee-52ca-4fcf-9066-1ff83cd08c8d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.213382 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edcee-52ca-4fcf-9066-1ff83cd08c8d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.213659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edcee-52ca-4fcf-9066-1ff83cd08c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.233498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-039edcee-52ca-4fcf-9066-1ff83cd08c8d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:01.234695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-039edcee-52ca-4fcf-9066-1ff83cd08c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:01.298561 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1209/2426] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:01.307168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950c2d-f2e4-4e9a-8757-bad03c0cf642 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.307451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950c2d-f2e4-4e9a-8757-bad03c0cf642 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.307733 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950c2d-f2e4-4e9a-8757-bad03c0cf642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.328320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-26950c2d-f2e4-4e9a-8757-bad03c0cf642 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:01.329525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-26950c2d-f2e4-4e9a-8757-bad03c0cf642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:01.338294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1218/2427] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:01.342500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edcee-52ca-4fcf-9066-1ff83cd08c8d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.352529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edcee-52ca-4fcf-9066-1ff83cd08c8d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.357044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.357639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.357918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.358189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.358829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:08:01.359132 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.360811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.369841 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:01.373501 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1210/2428] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:08:01.378494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950c2d-f2e4-4e9a-8757-bad03c0cf642 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.387720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950c2d-f2e4-4e9a-8757-bad03c0cf642 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.393290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.393886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.394166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.394436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.395064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.397281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.424074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1219/2429] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:08:01 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:08:01.428271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.438240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e72765-3d10-4985-99a9-906c992b2460 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.494268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.494868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.495149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.495422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.496081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.497994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.519895 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1211/2430] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:08:01 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:08:01.524294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.534360 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547cdbf-6abe-4684-8a62-e024d6541ac8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.538962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.539558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.539858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.540145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.540791 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.542384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.552323 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1220/2431] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:08:01.556353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.566795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e0ef-1ff6-4109-be2b-0733878d6371 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.576611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbfab5-1666-4c48-b9f3-e2cf0a801c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.577216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbfab5-1666-4c48-b9f3-e2cf0a801c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.577492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbfab5-1666-4c48-b9f3-e2cf0a801c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.577763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbfab5-1666-4c48-b9f3-e2cf0a801c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.578411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbfab5-1666-4c48-b9f3-e2cf0a801c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.580738 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbfab5-1666-4c48-b9f3-e2cf0a801c67 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.591616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1212/2432] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:08:01.595956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.607211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8664-9921-484f-8edc-b1e062159749 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.612381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.613022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.613301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.613575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] PATH_INFO: `/v3/projects/8f981425a83c4eb6a71891c756977764/users/59074dbc0ef44686a396b6aaaf19ea25/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.629810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f981425a83c4eb6a71891c756977764, user_id=59074dbc0ef44686a396b6aaaf19ea25, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.632502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.645549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1221/2433] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:08:01 2022] PUT /identity/v3/projects/8f981425a83c4eb6a71891c756977764/users/59074dbc0ef44686a396b6aaaf19ea25/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:08:01.651308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbdbd7-5566-4880-8017-7f50dac695c1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.651866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbdbd7-5566-4880-8017-7f50dac695c1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.652120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbdbd7-5566-4880-8017-7f50dac695c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.671461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-66bbdbd7-5566-4880-8017-7f50dac695c1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:01.672679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-66bbdbd7-5566-4880-8017-7f50dac695c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:01.731795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1213/2434] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:01.742186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:01.743872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:01.758626 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:01.804228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778be589-74d9-4dc4-aa3c-a37bc2849d2a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:01.807888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:01.808484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:01.808760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:01.809030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:01.818548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:01.820282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:01.829492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1222/2435] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:08:02.607938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d13d0-e013-453f-baeb-7abe64c5b0f8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:02.608248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d13d0-e013-453f-baeb-7abe64c5b0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:02.608527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d13d0-e013-453f-baeb-7abe64c5b0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:02.636009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d65d13d0-e013-453f-baeb-7abe64c5b0f8 None None] MFA Rules not processed for user `59074dbc0ef44686a396b6aaaf19ea25`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:02.637392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d65d13d0-e013-453f-baeb-7abe64c5b0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:02.703012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1214/2436] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:08:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:02.714739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:02.716379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:02.784260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef47f44-e974-4dd3-90a9-e43c42f20b57 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:02.787751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba950b5-a7ad-43e2-85b2-28a6925319ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:02.788396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba950b5-a7ad-43e2-85b2-28a6925319ed service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:02.788701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba950b5-a7ad-43e2-85b2-28a6925319ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:02.789017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba950b5-a7ad-43e2-85b2-28a6925319ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:02.797413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba950b5-a7ad-43e2-85b2-28a6925319ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:02.798826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba950b5-a7ad-43e2-85b2-28a6925319ed service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:02.807543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1223/2437] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:08:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:08:04.162324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:04.162835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:04.162931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:04.181949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] MFA Rules not processed for user `59074dbc0ef44686a396b6aaaf19ea25`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:04.183143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:04.190400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1215/2438] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:08:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:12.803562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=46 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:14.177153 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:22.263503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c53d-af41-4b21-941f-56409990e598 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:22.264013 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c53d-af41-4b21-941f-56409990e598 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:22.264106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c53d-af41-4b21-941f-56409990e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:22.285649 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c741c53d-af41-4b21-941f-56409990e598 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:22.286868 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c741c53d-af41-4b21-941f-56409990e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:22.344121 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1224/2439] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:08:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:22.363041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:22.367432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:22.450187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7352a1-0657-4baa-9930-0ce471027bd0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:22.455081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7d3b9-870e-4827-92dc-bf194475cdc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:22.455913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d3b9-870e-4827-92dc-bf194475cdc9 service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:22.456183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d3b9-870e-4827-92dc-bf194475cdc9 service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:22.456427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d3b9-870e-4827-92dc-bf194475cdc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:22.465544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7d3b9-870e-4827-92dc-bf194475cdc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:22.467101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7d3b9-870e-4827-92dc-bf194475cdc9 service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:22.475755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1216/2440] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:08:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:08:32.337499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=25 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:32.472745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=32 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:38.713588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aa2f7-df0a-467b-9d71-4245140ae325 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:38.714189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aa2f7-df0a-467b-9d71-4245140ae325 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:38.714189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aa2f7-df0a-467b-9d71-4245140ae325 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:38.715527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1225/2441] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:08:38.723085 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373f18-4002-4d2a-a4a3-287c49a8c042 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:38.723572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373f18-4002-4d2a-a4a3-287c49a8c042 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:38.723667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373f18-4002-4d2a-a4a3-287c49a8c042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:38.744250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-06373f18-4002-4d2a-a4a3-287c49a8c042 None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:38.745459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-06373f18-4002-4d2a-a4a3-287c49a8c042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:38.817789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1217/2442] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:08:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:08:38.830315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3aa2f7-df0a-467b-9d71-4245140ae325 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:38.833177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6f3aa2f7-df0a-467b-9d71-4245140ae325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:38.892268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3aa2f7-df0a-467b-9d71-4245140ae325 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:38.896345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8636-7805-4603-b344-8733f93dd47d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:38.896940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8636-7805-4603-b344-8733f93dd47d service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:38.897477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8636-7805-4603-b344-8733f93dd47d service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:38.897751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8636-7805-4603-b344-8733f93dd47d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:38.908064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8636-7805-4603-b344-8733f93dd47d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:38.910055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8636-7805-4603-b344-8733f93dd47d service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:38.920535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1226/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:08:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:08:38.989108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e6e2-f675-4611-840e-6c9198c488e7 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:38.989413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e6e2-f675-4611-840e-6c9198c488e7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:38.989684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e6e2-f675-4611-840e-6c9198c488e7 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:38.991030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1218/2444] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:08:38.995823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9446f-5ee8-4396-ba24-634fcc09295c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:38.996136 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9446f-5ee8-4396-ba24-634fcc09295c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:38.996411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9446f-5ee8-4396-ba24-634fcc09295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:39.015610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-21c9446f-5ee8-4396-ba24-634fcc09295c None None] MFA Rules not processed for user `b5c892640b3e409b9a99b4ca1d34a964`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:39.016837 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-21c9446f-5ee8-4396-ba24-634fcc09295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:39.023404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1227/2445] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:08:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:08:39.035618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190e6e2-f675-4611-840e-6c9198c488e7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:39.037326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c190e6e2-f675-4611-840e-6c9198c488e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:39.051052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190e6e2-f675-4611-840e-6c9198c488e7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:39.054487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607866d-09c5-4c7f-a607-4a7e4cf7c489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:39.055073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607866d-09c5-4c7f-a607-4a7e4cf7c489 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:39.055348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607866d-09c5-4c7f-a607-4a7e4cf7c489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:39.055614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607866d-09c5-4c7f-a607-4a7e4cf7c489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:39.062474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607866d-09c5-4c7f-a607-4a7e4cf7c489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:39.063921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607866d-09c5-4c7f-a607-4a7e4cf7c489 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:39.071678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1219/2446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:08:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:08:39.157638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd3dc-e444-4cdb-8814-cbd04a94c7ff None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:39.157930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd3dc-e444-4cdb-8814-cbd04a94c7ff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:39.158230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd3dc-e444-4cdb-8814-cbd04a94c7ff None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:39.160298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1228/2447] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:08:39.165531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ae62c-349e-4229-a02d-8a5f54cf1bd4 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:39.165849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ae62c-349e-4229-a02d-8a5f54cf1bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:39.166127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ae62c-349e-4229-a02d-8a5f54cf1bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:39.185616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7e6ae62c-349e-4229-a02d-8a5f54cf1bd4 None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:39.186797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7e6ae62c-349e-4229-a02d-8a5f54cf1bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:39.196177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1220/2448] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:08:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:39.207042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd3dc-e444-4cdb-8814-cbd04a94c7ff None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:39.217914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd3dc-e444-4cdb-8814-cbd04a94c7ff None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:39.222080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:39.222667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:39.222942 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:39.223217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:39.233231 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:39.234723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:39.245565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1229/2449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:08:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:08:42.063870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7be563-617e-42b6-a5d5-e84a1294b642 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:42.064352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7be563-617e-42b6-a5d5-e84a1294b642 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:42.064457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7be563-617e-42b6-a5d5-e84a1294b642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:42.084381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bf7be563-617e-42b6-a5d5-e84a1294b642 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:08:42.086386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bf7be563-617e-42b6-a5d5-e84a1294b642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:42.094057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1221/2450] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:08:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:08:42.106662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:42.117729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c969-745c-45a5-bca9-41230fd12a41 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:42.122950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:42.123551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:42.123870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:42.124148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:42.132537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:42.134054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:42.142194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1230/2451] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:08:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:08:52.078819 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=43 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:52.138853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=41 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:08:58.963530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7be563-617e-42b6-a5d5-e84a1294b642 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:08:58.973037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7be563-617e-42b6-a5d5-e84a1294b642 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:08:58.976946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:08:58.977538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:08:58.977817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:08:58.978089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:08:58.979772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:08:58.993286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:08:58.994782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:08:59.006302 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1222/2452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:08:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:00.200662 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:00.209916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3cc0-8df3-4b78-b942-a85790e8c0b5 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:00.213521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:00.214110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:00.214393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:00.214672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:00.216401 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:00.229384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:00.230801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:00.242748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1231/2453] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:09.000884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:09:10.237421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:09:13.169645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:13.171151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:13.185919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826f59-e319-42ab-800b-3480a299a13d service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:13.191377 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596eb8-ea80-4875-b092-6e4b6227a30c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:13.192084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596eb8-ea80-4875-b092-6e4b6227a30c service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:13.192481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596eb8-ea80-4875-b092-6e4b6227a30c service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:13.192962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596eb8-ea80-4875-b092-6e4b6227a30c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:13.194647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-70596eb8-ea80-4875-b092-6e4b6227a30c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:13.252963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596eb8-ea80-4875-b092-6e4b6227a30c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:13.254546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596eb8-ea80-4875-b092-6e4b6227a30c service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:13.263714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1223/2454] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:17.170064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:17.180385 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf6358-068f-4b76-8be5-6a5c02b83c3d service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:17.186069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:17.186688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:17.186963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:17.187233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:17.188949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:17.247224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:17.248835 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:17.258332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1232/2455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:19.101387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aadde7-6959-4170-aad0-f921fa16ae90 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:19.101908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aadde7-6959-4170-aad0-f921fa16ae90 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:19.102014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aadde7-6959-4170-aad0-f921fa16ae90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:19.121038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-49aadde7-6959-4170-aad0-f921fa16ae90 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:09:19.122284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-49aadde7-6959-4170-aad0-f921fa16ae90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:19.129450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1224/2456] 10.222.0.52 () {58 vars in 967 bytes} [Thu Oct 6 11:09:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:09:19.142708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:19.169182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526077f3-edba-4cf7-b0cc-3d3cdb25f800 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:19.179754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:19.180614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:19.181018 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:19.181260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:19.197635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:19.199518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:19.208750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1233/2457] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:09:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:09:29.116143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=31 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:09:29.205199 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=34 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:09:48.836856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:48.837365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:48.837513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:48.856899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] MFA Rules not processed for user `59074dbc0ef44686a396b6aaaf19ea25`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:09:48.858096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:48.865024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1225/2458] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:09:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:09:49.342668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:49.353578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7f083-e253-47da-b8ab-1abdbc6bdaab service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:49.357299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:49.357895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:49.358176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:49.358451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:49.360164 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:49.369565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:49.370974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:49.378881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1234/2459] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:49.696559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:49.698053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:49.770852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4adb0-2bc2-4bf6-9376-d23d7e16c9d8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:49.776279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:49.776877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:49.777156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:49.777428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:49.779098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:49.790284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:49.791871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:49.801458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1226/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:51.745718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:51.754826 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13239ecc-7b48-4b8d-b3bf-e34c14b8eff8 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:51.758620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acdcd2-8cca-4606-a47c-ed040919688e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:51.759223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acdcd2-8cca-4606-a47c-ed040919688e service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:51.759503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acdcd2-8cca-4606-a47c-ed040919688e service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:51.759774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acdcd2-8cca-4606-a47c-ed040919688e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:51.761787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-98acdcd2-8cca-4606-a47c-ed040919688e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:51.771709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acdcd2-8cca-4606-a47c-ed040919688e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:51.773433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acdcd2-8cca-4606-a47c-ed040919688e service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:51.782491 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1235/2461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:52.421754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:52.423264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:52.438709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bc741-a390-4cf2-b8ef-ac9f7b636f07 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:52.444087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:52.444680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:52.444955 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:52.445226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:52.447573 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:52.459621 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:52.461109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:52.472019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1227/2462] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:09:56.470703 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29f224-8616-49c1-8390-7fe39ba7874a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:56.471250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29f224-8616-49c1-8390-7fe39ba7874a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:56.471250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29f224-8616-49c1-8390-7fe39ba7874a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:56.489949 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5c29f224-8616-49c1-8390-7fe39ba7874a None None] MFA Rules not processed for user `f1815cf2b3f347eb8276769cc30cf116`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:09:56.491125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c29f224-8616-49c1-8390-7fe39ba7874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:56.498571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1236/2463] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:09:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:09:57.049591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:09:57.061275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584176e-717e-4da6-8fb1-1dcf11fcecc1 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:09:57.067652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:09:57.068305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:09:57.068583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:09:57.068858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:09:57.070917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:09:57.135149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:09:57.136707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:09:57.145967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1228/2464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:09:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:10:06.484531 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=15 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:10:07.141629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=69 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:00.713036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520ae55-6586-454d-84eb-479448f4502b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:00.713579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520ae55-6586-454d-84eb-479448f4502b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:00.713579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520ae55-6586-454d-84eb-479448f4502b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:00.734913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0520ae55-6586-454d-84eb-479448f4502b None None] MFA Rules not processed for user `56835f0c8dc0416d9fa70662e40c856d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:00.736183 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0520ae55-6586-454d-84eb-479448f4502b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:00.743289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1237/2465] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:00.999816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.012119 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a585b35-b6dd-4dbe-a9c2-43eddfe93f79 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.017660 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.018256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.018737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.019008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.020773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:01.034552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.036180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.049039 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1229/2466] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:11:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:11:01.067566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520ae55-6586-454d-84eb-479448f4502b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.081111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520ae55-6586-454d-84eb-479448f4502b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.086910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.087776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.088350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.089028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] PATH_INFO: `/v3/users/59074dbc0ef44686a396b6aaaf19ea25` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.090494 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59074dbc0ef44686a396b6aaaf19ea25)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.093514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.147948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59074dbc0ef44686a396b6aaaf19ea25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.160381 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59074dbc0ef44686a396b6aaaf19ea25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.167927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59074dbc0ef44686a396b6aaaf19ea25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.168658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59074dbc0ef44686a396b6aaaf19ea25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.176207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e04fbe2-1d01-4b66-9bba-877091661345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59074dbc0ef44686a396b6aaaf19ea25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.182510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1238/2467] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:11:01 2022] DELETE /identity/v3/users/59074dbc0ef44686a396b6aaaf19ea25 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:11:01.373540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.384878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6947b-b5d1-4473-b8a2-1e022baf685f service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.453001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.453739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.454123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.454629 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] PATH_INFO: `/v3/projects/8f981425a83c4eb6a71891c756977764` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.456393 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f981425a83c4eb6a71891c756977764)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.458808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.519569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f981425a83c4eb6a71891c756977764', 'request_id': 'req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.525448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f981425a83c4eb6a71891c756977764', 'request_id': 'req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.533261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f981425a83c4eb6a71891c756977764', 'request_id': 'req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.547481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] The token cache is being invalidate because project 8f981425a83c4eb6a71891c756977764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:11:01.547773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.548443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5b4c9133-7c8b-48fc-8bd3-44ff8754d076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:01.549904 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1230/2468] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:11:01 2022] DELETE /identity/v3/projects/8f981425a83c4eb6a71891c756977764 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:11:01.554696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9816d-2c04-4415-b47e-d3369635f2fc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.555207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9816d-2c04-4415-b47e-d3369635f2fc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.555498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9816d-2c04-4415-b47e-d3369635f2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.576920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4bd9816d-2c04-4415-b47e-d3369635f2fc None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:01.578517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4bd9816d-2c04-4415-b47e-d3369635f2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:01.641159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1239/2469] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:01.649133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec04ffe-1840-4f35-af5d-f3ab65ca2605 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.649424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec04ffe-1840-4f35-af5d-f3ab65ca2605 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.649698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec04ffe-1840-4f35-af5d-f3ab65ca2605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.671321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7ec04ffe-1840-4f35-af5d-f3ab65ca2605 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:01.672601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7ec04ffe-1840-4f35-af5d-f3ab65ca2605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:01.686925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1231/2470] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:01.690781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9816d-2c04-4415-b47e-d3369635f2fc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.701879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9816d-2c04-4415-b47e-d3369635f2fc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.706654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.707465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.707743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.708052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.708705 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:11:01.709028 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.710632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.726914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1240/2471] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:11:01.729044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec04ffe-1840-4f35-af5d-f3ab65ca2605 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.738847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec04ffe-1840-4f35-af5d-f3ab65ca2605 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.744067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.744668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.744950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.745227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.745889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.748431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.776943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1232/2472] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:11:01 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:11:01.781823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.792373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ab339-ebb9-4ca3-89b0-feecfe95efcf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.844559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:01.856732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.857295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.857957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.858287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.859124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.861345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.885839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1241/2473] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:11:01 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:11:01.887340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.899647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aac04c-22aa-47ea-9a01-5de0b1eadc68 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.905044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.905764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.906044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.906395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.907047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.908713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.919977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1233/2474] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:11:01.924038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.934489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1575a9-e027-4b88-b449-901b4f4a6996 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.940114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b277a-1dcd-4108-aa4c-3d06d5d2dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.940719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b277a-1dcd-4108-aa4c-3d06d5d2dfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.940999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b277a-1dcd-4108-aa4c-3d06d5d2dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.941275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b277a-1dcd-4108-aa4c-3d06d5d2dfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.941916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b277a-1dcd-4108-aa4c-3d06d5d2dfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.943493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b277a-1dcd-4108-aa4c-3d06d5d2dfe1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:01.955079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1242/2475] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:11:01.959330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:01.969553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139d032-3e2a-48c1-a4a1-41827bd5e49d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:01.974166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:01.974801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:01.975077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:01.975348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] PATH_INFO: `/v3/projects/ca311e63675241eaaef42a8154e905ac/users/5a5f8ee0a7fd4d7a9d34e211dabab48a/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:01.989882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca311e63675241eaaef42a8154e905ac, user_id=5a5f8ee0a7fd4d7a9d34e211dabab48a, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:01.992580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:02.006883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1234/2476] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:11:01 2022] PUT /identity/v3/projects/ca311e63675241eaaef42a8154e905ac/users/5a5f8ee0a7fd4d7a9d34e211dabab48a/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:11:02.011915 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f2b6e-4c90-4ea6-abf6-8b6d7695e5c1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:02.012211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f2b6e-4c90-4ea6-abf6-8b6d7695e5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:02.012492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f2b6e-4c90-4ea6-abf6-8b6d7695e5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:02.033007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d67f2b6e-4c90-4ea6-abf6-8b6d7695e5c1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:02.034217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d67f2b6e-4c90-4ea6-abf6-8b6d7695e5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:02.099749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1243/2477] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:02.110367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:02.112096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:02.127521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:02.175213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc2acc-5d02-4f26-b96a-dcab6dd6dc7d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:02.179059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:02.179661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:02.179983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:02.180243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:02.191006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:02.192507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:02.203050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1235/2478] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:11:07.842118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae93c93-4336-4753-a815-523af9a52daf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:07.843053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae93c93-4336-4753-a815-523af9a52daf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:07.843343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae93c93-4336-4753-a815-523af9a52daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:07.877857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3ae93c93-4336-4753-a815-523af9a52daf None None] MFA Rules not processed for user `5a5f8ee0a7fd4d7a9d34e211dabab48a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:07.879384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3ae93c93-4336-4753-a815-523af9a52daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:07.955878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1244/2479] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:07.959967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:07.970418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58c942-d8bc-4efe-b1c6-b9ee7ded467a service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:07.976127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:07.976745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:07.977028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:07.977303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:07.977948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:07.980327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:08.014886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1236/2480] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:11:07 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:11:08.019161 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae93c93-4336-4753-a815-523af9a52daf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:08.034908 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae93c93-4336-4753-a815-523af9a52daf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:08.096536 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:08.097290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:08.097611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:08.097947 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:08.098795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:08.100896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:08.110911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:08.124429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1245/2481] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:11:08 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:11:08.128368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:08.139006 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011d25e-f06a-41dc-b2f9-46ec05b13f77 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:08.143708 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:08.144330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:08.144616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:08.144886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:08.145523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:08.147371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:08.158040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1237/2482] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:11:08.162097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:08.172538 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40136073-8226-488c-aff9-2748f340ea54 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:08.177084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:08.177721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:08.177997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:08.178267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] PATH_INFO: `/v3/projects/6bd72824226c40f0bb4f52f870ec41f4/users/eb6df62737274d0da97137b2ada2d6fb/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:08.193201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd72824226c40f0bb4f52f870ec41f4, user_id=eb6df62737274d0da97137b2ada2d6fb, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:08.195916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:08.207793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1246/2483] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:11:08 2022] PUT /identity/v3/projects/6bd72824226c40f0bb4f52f870ec41f4/users/eb6df62737274d0da97137b2ada2d6fb/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:11:08.211645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:08.222241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5587e7-050c-40cb-9d5b-24c55d73b2f3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:08.280259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:08.280859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:08.281137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:08.281408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:08.282045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:08.283632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:08.293193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1238/2484] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:11:08.298192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:08.308392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a0722-2e4d-4e4d-9a07-ed90e39ce964 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:08.313816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf315a4-7f79-4996-9731-4420e6668d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:08.314455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf315a4-7f79-4996-9731-4420e6668d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:08.314732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf315a4-7f79-4996-9731-4420e6668d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:08.315002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf315a4-7f79-4996-9731-4420e6668d9b admin admin] PATH_INFO: `/v3/projects/6bd72824226c40f0bb4f52f870ec41f4/users/eb6df62737274d0da97137b2ada2d6fb/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:08.316930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf315a4-7f79-4996-9731-4420e6668d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd72824226c40f0bb4f52f870ec41f4, user_id=eb6df62737274d0da97137b2ada2d6fb, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:08.319561 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf315a4-7f79-4996-9731-4420e6668d9b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:08.329865 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1247/2485] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:11:08 2022] PUT /identity/v3/projects/6bd72824226c40f0bb4f52f870ec41f4/users/eb6df62737274d0da97137b2ada2d6fb/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:11:12.984991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:12.988057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:13.141918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29cabb-4005-4a80-b9a9-44923551526e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:13.191408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:13.319671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:13.320105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:13.320371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:13.320574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:13.325223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:13.343711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:13.345485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:13.360147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1239/2486] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:11:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:11:13.619440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cb5a-babd-4d9a-ac04-51cd21cfe4d6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:13.620081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cb5a-babd-4d9a-ac04-51cd21cfe4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:13.620081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cb5a-babd-4d9a-ac04-51cd21cfe4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:13.651990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ab01cb5a-babd-4d9a-ac04-51cd21cfe4d6 None None] MFA Rules not processed for user `eb6df62737274d0da97137b2ada2d6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:13.653308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab01cb5a-babd-4d9a-ac04-51cd21cfe4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:13.759987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1248/2487] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:13.769172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:13.771735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:13.862982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f407-1467-4212-b6a0-123b25a4d0e1 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:13.866739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:13.867539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:13.867696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:13.868035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:13.949634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:13.951779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:13.961187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1240/2488] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:11:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:11:14.096930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836e930-e4cb-4c69-9453-0fecc9a8f492 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:14.097572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836e930-e4cb-4c69-9453-0fecc9a8f492 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:14.097965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836e930-e4cb-4c69-9453-0fecc9a8f492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:14.118515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7836e930-e4cb-4c69-9453-0fecc9a8f492 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:14.119735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7836e930-e4cb-4c69-9453-0fecc9a8f492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:14.178807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1249/2489] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:14.229856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:14.231238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:14.294853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:14.310542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b4524-1d16-4b17-a541-82de353aff18 service glance] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:14.315724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6d3e-685e-4338-abd9-2bbab8649d53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:14.316414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6d3e-685e-4338-abd9-2bbab8649d53 service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:14.316695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6d3e-685e-4338-abd9-2bbab8649d53 service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:14.316966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6d3e-685e-4338-abd9-2bbab8649d53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:14.326238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6d3e-685e-4338-abd9-2bbab8649d53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:14.327946 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6d3e-685e-4338-abd9-2bbab8649d53 service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:14.336604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1241/2490] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:11:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:11:23.458985 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac6ee5-a86b-4532-80a3-a37eff581e55 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:23.459696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac6ee5-a86b-4532-80a3-a37eff581e55 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:23.459696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac6ee5-a86b-4532-80a3-a37eff581e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:23.482523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b8ac6ee5-a86b-4532-80a3-a37eff581e55 None None] MFA Rules not processed for user `5a5f8ee0a7fd4d7a9d34e211dabab48a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:23.483722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b8ac6ee5-a86b-4532-80a3-a37eff581e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:23.490641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1250/2491] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:24.332908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:29.193975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:29.194702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:29.194702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:29.215481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:29.217326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:29.225807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1242/2492] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:11:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:29.239041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac6ee5-a86b-4532-80a3-a37eff581e55 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:29.251404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac6ee5-a86b-4532-80a3-a37eff581e55 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:29.260913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e0ea1-b44d-42d2-9990-c588331171dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:29.261883 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e0ea1-b44d-42d2-9990-c588331171dc service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:29.262740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e0ea1-b44d-42d2-9990-c588331171dc service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:29.262740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e0ea1-b44d-42d2-9990-c588331171dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:29.273076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e0ea1-b44d-42d2-9990-c588331171dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:29.274787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e0ea1-b44d-42d2-9990-c588331171dc service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:29.287219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1251/2493] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:11:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:11:39.209730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:39.281176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=4,keystone:SELECT=85 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:42.661325 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:42.662975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:42.684443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210f7f3-89c7-4f72-9534-f57bb9fbf286 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:42.691468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:42.692456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:42.692801 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:42.693152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:42.695061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:42.713665 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:42.715259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:42.726328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1243/2494] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:11:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:11:49.496418 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:49.496903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:49.497008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:49.517784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] MFA Rules not processed for user `1bb1115590ac45fdbcdc6d3f52aa6921`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:11:49.519474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:49.580732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1252/2495] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:11:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:11:52.721844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:55.838746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:55.893775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322d90f-dc63-447a-b9a1-95aab096ba38 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:55.897701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:55.898297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:55.898577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:55.898851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:55.900619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:56.093032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:56.095319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:56.115342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1244/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:11:59.572941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=25 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:11:59.627100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:59.628634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:11:59.644652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491adf9-f4d6-41d0-90e8-5218d4820214 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:59.649431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:59.650061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:59.650349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:59.650622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] PATH_INFO: `/v3/users/f1815cf2b3f347eb8276769cc30cf116` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:59.651744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1815cf2b3f347eb8276769cc30cf116)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:59.653740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:11:59.700737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1815cf2b3f347eb8276769cc30cf116'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:59.712689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1815cf2b3f347eb8276769cc30cf116'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:59.720332 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1815cf2b3f347eb8276769cc30cf116'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:59.721010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1815cf2b3f347eb8276769cc30cf116'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:59.728004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2fadcf-bfc9-4187-a7ba-13cb691b6cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1815cf2b3f347eb8276769cc30cf116'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:11:59.733841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1253/2497] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:11:59 2022] DELETE /identity/v3/users/f1815cf2b3f347eb8276769cc30cf116 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:11:59.909781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:11:59.920171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b4fd-68f4-46b3-9340-599fc7d0dea1 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:11:59.977842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:11:59.978464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:11:59.978744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:11:59.979018 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] PATH_INFO: `/v3/projects/4ebb258de32f40c68f86e4b401efde5d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:11:59.980150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ebb258de32f40c68f86e4b401efde5d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:11:59.982083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.028036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ebb258de32f40c68f86e4b401efde5d', 'request_id': 'req-1183117e-de70-49ae-b0fc-a6e848802850'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:00.029862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:00.034687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ebb258de32f40c68f86e4b401efde5d', 'request_id': 'req-1183117e-de70-49ae-b0fc-a6e848802850'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:00.041295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ebb258de32f40c68f86e4b401efde5d', 'request_id': 'req-1183117e-de70-49ae-b0fc-a6e848802850'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:00.054386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] The token cache is being invalidate because project 4ebb258de32f40c68f86e4b401efde5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:12:00.054701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:00.055374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1183117e-de70-49ae-b0fc-a6e848802850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:00.056896 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1245/2498] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:11:59 2022] DELETE /identity/v3/projects/4ebb258de32f40c68f86e4b401efde5d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:00.061988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd4a2c-923b-4193-bd7b-a211e030ec1b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.062283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd4a2c-923b-4193-bd7b-a211e030ec1b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.062562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd4a2c-923b-4193-bd7b-a211e030ec1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.082635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6bbd4a2c-923b-4193-bd7b-a211e030ec1b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:00.083906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6bbd4a2c-923b-4193-bd7b-a211e030ec1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:00.145201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1254/2499] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:12:00.153913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f4301-02ae-4dd2-a4cb-9a6d5611fc71 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.154195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f4301-02ae-4dd2-a4cb-9a6d5611fc71 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.154467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f4301-02ae-4dd2-a4cb-9a6d5611fc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.175149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-928f4301-02ae-4dd2-a4cb-9a6d5611fc71 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:00.176372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-928f4301-02ae-4dd2-a4cb-9a6d5611fc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:00.184226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1246/2500] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:00.188412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd4a2c-923b-4193-bd7b-a211e030ec1b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.198019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd4a2c-923b-4193-bd7b-a211e030ec1b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.202551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.203674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.203674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.203921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.204483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:12:00.204891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.206354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.218288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1255/2501] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:12:00.223068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f4301-02ae-4dd2-a4cb-9a6d5611fc71 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.232225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f4301-02ae-4dd2-a4cb-9a6d5611fc71 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.237718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.238338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.238642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.238908 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.239670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.241600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.269680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1247/2502] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:12:00 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:12:00.280371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.291022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa90c0-9674-4515-9142-a2d781226508 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.346803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.347412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.347690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.347990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.348631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.350537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.358587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:00.376752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1256/2503] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:12:00 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:00.377981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.392748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ef9d0-70be-414d-8bca-373782232835 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.397868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.398542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.398891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.399231 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.399988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.401670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.412496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1248/2504] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:00.416566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.427217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659016e-bc4a-4314-813d-be954f10f166 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.431819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066322b9-fd56-4ac8-a042-c2d12dd039f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.432441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066322b9-fd56-4ac8-a042-c2d12dd039f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.432723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066322b9-fd56-4ac8-a042-c2d12dd039f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.433001 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066322b9-fd56-4ac8-a042-c2d12dd039f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.433640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066322b9-fd56-4ac8-a042-c2d12dd039f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.435218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066322b9-fd56-4ac8-a042-c2d12dd039f0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.445717 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1257/2505] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:00.449655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.460112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11edc2-d550-4fb6-b4c8-3891326dcb59 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.464714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.465361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.465641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.465917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] PATH_INFO: `/v3/projects/fd7bba8e07844dec9b462de5b46ba831/users/7f0c9bfdc1bb4918950c53f18cf68a79/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.482417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7bba8e07844dec9b462de5b46ba831, user_id=7f0c9bfdc1bb4918950c53f18cf68a79, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.485220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.498211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1249/2506] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:12:00 2022] PUT /identity/v3/projects/fd7bba8e07844dec9b462de5b46ba831/users/7f0c9bfdc1bb4918950c53f18cf68a79/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:00.508444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6379-58b3-4bb1-9910-622cf9722247 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.508731 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6379-58b3-4bb1-9910-622cf9722247 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.509008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6379-58b3-4bb1-9910-622cf9722247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.538465 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c90b6379-58b3-4bb1-9910-622cf9722247 None None] MFA Rules not processed for user `7f0c9bfdc1bb4918950c53f18cf68a79`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:00.539634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c90b6379-58b3-4bb1-9910-622cf9722247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:00.603715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1258/2507] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:00.608238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.670714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc35314-8976-4790-b379-01a6eb0657ab admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.675547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.676167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.676445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.676715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.677331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.679242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.705892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1250/2508] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:12:00 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:00.709933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b6379-58b3-4bb1-9910-622cf9722247 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.721671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b6379-58b3-4bb1-9910-622cf9722247 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.780194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.780807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.781073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.781345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.782553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.784503 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.831030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1259/2509] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:12:00 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:00.834897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.845022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84547c-2739-4b85-b0f4-7da51a29a7cd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.849689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.850303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.850578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.850843 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.851473 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.853088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.863526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1251/2510] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:00.867589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.894992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596de2a-6d40-4f28-ab35-5e1cc490da64 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.921308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:00.931334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ca517-b0ef-42f4-bfd7-b3b26e992450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:00.931980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ca517-b0ef-42f4-bfd7-b3b26e992450 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:00.932265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ca517-b0ef-42f4-bfd7-b3b26e992450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:00.932567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ca517-b0ef-42f4-bfd7-b3b26e992450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:00.933213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ca517-b0ef-42f4-bfd7-b3b26e992450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:00.934852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ca517-b0ef-42f4-bfd7-b3b26e992450 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:00.945122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1260/2511] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:00.949438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:00.994769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e34024-418d-4e45-8252-ac967d2aafa3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:00.999585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.000776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.001242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.001523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] PATH_INFO: `/v3/projects/a594399f671a490e9698b63526305aa4/users/6e466c81f142492ab48c2ac553299057/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.018431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a594399f671a490e9698b63526305aa4, user_id=6e466c81f142492ab48c2ac553299057, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.021295 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.023825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:01.036663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1252/2512] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:12:00 2022] PUT /identity/v3/projects/a594399f671a490e9698b63526305aa4/users/6e466c81f142492ab48c2ac553299057/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:12:01.047474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683596b-14b2-4249-81c7-8517c149a547 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.047776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683596b-14b2-4249-81c7-8517c149a547 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.048110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683596b-14b2-4249-81c7-8517c149a547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.077689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8683596b-14b2-4249-81c7-8517c149a547 None None] MFA Rules not processed for user `6e466c81f142492ab48c2ac553299057`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:01.079737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8683596b-14b2-4249-81c7-8517c149a547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:01.146315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1261/2513] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:01.151549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:01.219929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ef963-1db5-4e2d-8dde-c6bee28f1f4d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:01.225286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.225885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.226160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.226430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.227057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.228985 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.255618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1253/2514] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:12:01 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:01.259813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683596b-14b2-4249-81c7-8517c149a547 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:01.270936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683596b-14b2-4249-81c7-8517c149a547 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:01.328547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.329352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.330999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.331343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.332033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.333945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.356595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1262/2515] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:12:01 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:01.361467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:01.371549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19a94-6894-4ee8-80a6-89cbc9907dfb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:01.376794 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.377387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.377663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.377931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.378565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.380162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.391527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1254/2516] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:01.395586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:01.406482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19cb0a-6034-490a-b870-05512351332d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:01.411622 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.412343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.412630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.412906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] PATH_INFO: `/v3/projects/dbd2c7b7380f484eb5026c7ce67a6c2b/users/c3c9703671ed42d4aa452f776873ee0d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.430711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd2c7b7380f484eb5026c7ce67a6c2b, user_id=c3c9703671ed42d4aa452f776873ee0d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.433534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.448895 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1263/2517] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:12:01 2022] PUT /identity/v3/projects/dbd2c7b7380f484eb5026c7ce67a6c2b/users/c3c9703671ed42d4aa452f776873ee0d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:01.452745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:01.463127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dfa2d-0aec-480a-ace2-2e181cff438b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:01.519109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6008987-706f-4dc6-9f07-ea0c710e096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.519709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6008987-706f-4dc6-9f07-ea0c710e096b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.520025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6008987-706f-4dc6-9f07-ea0c710e096b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.520299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6008987-706f-4dc6-9f07-ea0c710e096b admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.520935 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6008987-706f-4dc6-9f07-ea0c710e096b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.522789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6008987-706f-4dc6-9f07-ea0c710e096b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.532238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1255/2518] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:01.536413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:01.546523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad27c8-35c2-47d6-877e-622ba02fcf55 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:01.551680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.552366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.552666 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.552929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] PATH_INFO: `/v3/projects/dbd2c7b7380f484eb5026c7ce67a6c2b/users/c3c9703671ed42d4aa452f776873ee0d/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.554851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd2c7b7380f484eb5026c7ce67a6c2b, user_id=c3c9703671ed42d4aa452f776873ee0d, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.557522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.568263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1264/2519] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:12:01 2022] PUT /identity/v3/projects/dbd2c7b7380f484eb5026c7ce67a6c2b/users/c3c9703671ed42d4aa452f776873ee0d/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:01.579887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.580177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.580455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.608032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] MFA Rules not processed for user `c3c9703671ed42d4aa452f776873ee0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:01.609227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:01.659374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:01.678643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1256/2520] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:01.689444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:01.691151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:01.723175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:01.765257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977c32-1dc8-4480-96cd-320d43d63366 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:01.769047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:01.769646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:01.769920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:01.770190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:01.779186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:01.780623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:01.790450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1265/2521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:02.284639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:02.286002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:02.298030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7df3d-1f61-4609-8bd4-5a32a95a21ae None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:02.301531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:02.302114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:02.302388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:02.302656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:02.357441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:02.358850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:02.368342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1257/2522] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:04.936992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:04.938900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:05.009204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc8603-af5f-4a8f-8b17-bc28247733b0 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:05.013256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:05.014059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:05.014495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:05.015016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:05.018809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:05.108529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:05.112455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:05.126627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1266/2523] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:06.554179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:06.563125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bb185-3c3d-4ecc-bf13-0b4a4ad0a3de service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:06.567163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b547637-ee67-4c03-9364-0a25fddd4626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:06.567864 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b547637-ee67-4c03-9364-0a25fddd4626 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:06.568249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b547637-ee67-4c03-9364-0a25fddd4626 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:06.568622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b547637-ee67-4c03-9364-0a25fddd4626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:06.628722 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b547637-ee67-4c03-9364-0a25fddd4626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:06.630625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b547637-ee67-4c03-9364-0a25fddd4626 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:06.639236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1258/2524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:09.012912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:09.108953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95f35-a13a-44c4-b9df-4ad05da26fab service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:09.116958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:09.117623 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:09.118008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:09.118286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] PATH_INFO: `/v3/users/7f0c9bfdc1bb4918950c53f18cf68a79` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:09.119457 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0c9bfdc1bb4918950c53f18cf68a79)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:09.122156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:09.129529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:09.216173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0c9bfdc1bb4918950c53f18cf68a79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.239193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0c9bfdc1bb4918950c53f18cf68a79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.252694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0c9bfdc1bb4918950c53f18cf68a79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.254944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0c9bfdc1bb4918950c53f18cf68a79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.266155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0c9bfdc1bb4918950c53f18cf68a79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.278303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1267/2525] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:12:09 2022] DELETE /identity/v3/users/7f0c9bfdc1bb4918950c53f18cf68a79 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:09.280022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137fac-1033-4657-8456-06736a450062 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:09.280975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137fac-1033-4657-8456-06736a450062 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:09.281911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137fac-1033-4657-8456-06736a450062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:09.313342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-52137fac-1033-4657-8456-06736a450062 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:09.314762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-52137fac-1033-4657-8456-06736a450062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:09.384518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1259/2526] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:09.394258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:09.395959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:09.477384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4959e-94d8-45f8-b309-e1021feb59f2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:09.481335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:09.482226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:09.482359 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:09.482781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:09.492517 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:09.494089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:09.508913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1268/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:09.783081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137fac-1033-4657-8456-06736a450062 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:09.784516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-52137fac-1033-4657-8456-06736a450062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:09.797678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137fac-1033-4657-8456-06736a450062 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:09.802390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:09.803509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:09.803790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:09.804089 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] PATH_INFO: `/v3/projects/fd7bba8e07844dec9b462de5b46ba831` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:09.805209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7bba8e07844dec9b462de5b46ba831)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:09.814842 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:09.831459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:09.862614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7bba8e07844dec9b462de5b46ba831', 'request_id': 'req-2ef17aa5-a003-4080-903c-fb3e562a61b6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.868168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7bba8e07844dec9b462de5b46ba831', 'request_id': 'req-2ef17aa5-a003-4080-903c-fb3e562a61b6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.874468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7bba8e07844dec9b462de5b46ba831', 'request_id': 'req-2ef17aa5-a003-4080-903c-fb3e562a61b6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.886530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] The token cache is being invalidate because project fd7bba8e07844dec9b462de5b46ba831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:12:09.886817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.887440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:09.888938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1260/2528] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:12:09 2022] DELETE /identity/v3/projects/fd7bba8e07844dec9b462de5b46ba831 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:09.893918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:09.905382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a05cc8-4c55-46e9-afae-8fbde1169c97 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:09.965359 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:09.966039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:09.966320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:09.966591 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] PATH_INFO: `/v3/users/6e466c81f142492ab48c2ac553299057` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:09.967688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e466c81f142492ab48c2ac553299057)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:09.969674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:10.002696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:10.018689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e466c81f142492ab48c2ac553299057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.030767 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e466c81f142492ab48c2ac553299057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.037929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e466c81f142492ab48c2ac553299057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.038542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e466c81f142492ab48c2ac553299057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.045333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e466c81f142492ab48c2ac553299057'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.051184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1269/2529] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:12:09 2022] DELETE /identity/v3/users/6e466c81f142492ab48c2ac553299057 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:10.319630 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:10.321169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:10.334975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17aa5-a003-4080-903c-fb3e562a61b6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:10.392844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:10.393479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:10.393764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:10.394041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] PATH_INFO: `/v3/projects/a594399f671a490e9698b63526305aa4` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:10.395160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a594399f671a490e9698b63526305aa4)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:10.397373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:10.444877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a594399f671a490e9698b63526305aa4', 'request_id': 'req-aa248f0f-3c77-463c-bb39-6464ec1318fc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.450288 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a594399f671a490e9698b63526305aa4', 'request_id': 'req-aa248f0f-3c77-463c-bb39-6464ec1318fc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.457260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a594399f671a490e9698b63526305aa4', 'request_id': 'req-aa248f0f-3c77-463c-bb39-6464ec1318fc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.469830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] The token cache is being invalidate because project a594399f671a490e9698b63526305aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:12:10.470286 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.470919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.472476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1261/2530] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:12:10 2022] DELETE /identity/v3/projects/a594399f671a490e9698b63526305aa4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:10.476653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:10.478132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:10.490964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86c7c-95f9-458d-9039-4608ab3ef11d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:10.546775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:10.547398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:10.547681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:10.547996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] PATH_INFO: `/v3/users/c3c9703671ed42d4aa452f776873ee0d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:10.549195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c9703671ed42d4aa452f776873ee0d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:10.551156 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:10.596907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c9703671ed42d4aa452f776873ee0d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.608846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c9703671ed42d4aa452f776873ee0d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.616214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c9703671ed42d4aa452f776873ee0d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.616833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c9703671ed42d4aa452f776873ee0d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.623688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb448713-a7b7-4792-9ebe-86257254d0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c9703671ed42d4aa452f776873ee0d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:10.630426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1270/2531] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:12:10 2022] DELETE /identity/v3/users/c3c9703671ed42d4aa452f776873ee0d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:10.905109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:10.906659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:10.919189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248f0f-3c77-463c-bb39-6464ec1318fc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:10.951109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:10.978825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:10.979458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:10.979737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:10.980040 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] PATH_INFO: `/v3/projects/dbd2c7b7380f484eb5026c7ce67a6c2b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:10.981154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd2c7b7380f484eb5026c7ce67a6c2b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:10.983073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.030498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd2c7b7380f484eb5026c7ce67a6c2b', 'request_id': 'req-6bd5e989-ab86-41fd-831a-f28751b63754'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.036223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd2c7b7380f484eb5026c7ce67a6c2b', 'request_id': 'req-6bd5e989-ab86-41fd-831a-f28751b63754'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.042448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd2c7b7380f484eb5026c7ce67a6c2b', 'request_id': 'req-6bd5e989-ab86-41fd-831a-f28751b63754'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.055224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] The token cache is being invalidate because project dbd2c7b7380f484eb5026c7ce67a6c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:12:11.055512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.056163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd5e989-ab86-41fd-831a-f28751b63754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.057628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1262/2532] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:12:10 2022] DELETE /identity/v3/projects/dbd2c7b7380f484eb5026c7ce67a6c2b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:11.065438 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897774-8625-4dbf-9ace-5f7f28049b3c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.065936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897774-8625-4dbf-9ace-5f7f28049b3c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.066039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897774-8625-4dbf-9ace-5f7f28049b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.105885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:11.119858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a2897774-8625-4dbf-9ace-5f7f28049b3c None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:11.121082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a2897774-8625-4dbf-9ace-5f7f28049b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:11.180698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1271/2533] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:11.189492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066a476-e991-4288-8ba4-8da3c3c0bdd8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.189776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066a476-e991-4288-8ba4-8da3c3c0bdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.190054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066a476-e991-4288-8ba4-8da3c3c0bdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.214873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9066a476-e991-4288-8ba4-8da3c3c0bdd8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:11.216137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9066a476-e991-4288-8ba4-8da3c3c0bdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:11.225977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1263/2534] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:11.230294 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2897774-8625-4dbf-9ace-5f7f28049b3c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.240448 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2897774-8625-4dbf-9ace-5f7f28049b3c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.244905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.245489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.245771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.246039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.246673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:12:11.246994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.248610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.259907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1272/2535] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:12:11.264289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066a476-e991-4288-8ba4-8da3c3c0bdd8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.274294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066a476-e991-4288-8ba4-8da3c3c0bdd8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.279125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.279698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.280087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.280360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.280988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.283496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.309536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1264/2536] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:12:11 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:12:11.314589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.324628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683499b-aeec-4978-8495-cca737666de3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.381405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.381924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.382204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.382475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.383091 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.385048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.409192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1273/2537] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:12:11 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:11.413287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.423674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29445ce5-d6af-4bd8-8112-7c9186c1532e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.428214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.428799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.429081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.429350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.429983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.431555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.442233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1265/2538] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:11.445994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.456521 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae86b5e-86ce-41fb-9655-c0995d9f749b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.461054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cc41e-5d47-4dc2-ae94-190bc958bdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.461635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cc41e-5d47-4dc2-ae94-190bc958bdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.461912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cc41e-5d47-4dc2-ae94-190bc958bdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.462184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cc41e-5d47-4dc2-ae94-190bc958bdd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.462816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cc41e-5d47-4dc2-ae94-190bc958bdd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.464434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cc41e-5d47-4dc2-ae94-190bc958bdd9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.474856 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1274/2539] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:11.478746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.489230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ff616-2b0b-49af-bbc4-63106b8930c6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.493789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.494429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.494703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.494971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] PATH_INFO: `/v3/projects/8647945f5f4f4a508d13e1802bbb0aaa/users/dcc992ec15ce498692eaa7475c66ba01/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.509684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8647945f5f4f4a508d13e1802bbb0aaa, user_id=dcc992ec15ce498692eaa7475c66ba01, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.512358 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.526184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1266/2540] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:12:11 2022] PUT /identity/v3/projects/8647945f5f4f4a508d13e1802bbb0aaa/users/dcc992ec15ce498692eaa7475c66ba01/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:11.538539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713592-98a2-4c8b-984c-717ed45577fa None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.538833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713592-98a2-4c8b-984c-717ed45577fa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.539141 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713592-98a2-4c8b-984c-717ed45577fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.566755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-93713592-98a2-4c8b-984c-717ed45577fa None None] MFA Rules not processed for user `dcc992ec15ce498692eaa7475c66ba01`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:11.568003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93713592-98a2-4c8b-984c-717ed45577fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:11.627215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:11.633454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1275/2541] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:11.645828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.647808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:11.705222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd0086-73b7-47f2-80c2-61faa3b8e921 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.708853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.709442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.709721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.709992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.718098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.720003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.728558 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1267/2542] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:11.752339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93713592-98a2-4c8b-984c-717ed45577fa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.821005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93713592-98a2-4c8b-984c-717ed45577fa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:11.825795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:11.826419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.826696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.826983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] PATH_INFO: `/v3/users/dcc992ec15ce498692eaa7475c66ba01` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.828144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc992ec15ce498692eaa7475c66ba01)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:11.830417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:11.837731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:11.839087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:11.843113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:11.882041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc992ec15ce498692eaa7475c66ba01'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.894493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc992ec15ce498692eaa7475c66ba01'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.902327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc992ec15ce498692eaa7475c66ba01'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.903266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc992ec15ce498692eaa7475c66ba01'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.910437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7a940244-9d44-46b1-9599-4c502feb88da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc992ec15ce498692eaa7475c66ba01'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:11.916518 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1276/2543] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:12:11 2022] DELETE /identity/v3/users/dcc992ec15ce498692eaa7475c66ba01 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:11.921132 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:11.921468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:11.921838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:11.941840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:11.943153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:11.946035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a95c9e-4867-4ff1-af15-b390928abca1 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.003160 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1277/2544] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:12.012480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.012902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.013087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.013364 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.013637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.014227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:12.015399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:12.027177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.028790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.034840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:12.038948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1268/2545] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:12:12.075861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19934-2f23-44a9-aba9-18624441c0da None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.079675 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcb1dc-23c4-4d2d-89ec-16b8d41c20e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.080312 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb1dc-23c4-4d2d-89ec-16b8d41c20e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.080597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb1dc-23c4-4d2d-89ec-16b8d41c20e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.080867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb1dc-23c4-4d2d-89ec-16b8d41c20e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.089855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcb1dc-23c4-4d2d-89ec-16b8d41c20e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.091279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcb1dc-23c4-4d2d-89ec-16b8d41c20e6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.101148 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1278/2546] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:12:12.377092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.378525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:12.392066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620c33-9a56-431a-aec4-7ade0f2e0691 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.396773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.397667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.397948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.398221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] PATH_INFO: `/v3/projects/8647945f5f4f4a508d13e1802bbb0aaa` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.399356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8647945f5f4f4a508d13e1802bbb0aaa)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.401308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.449254 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8647945f5f4f4a508d13e1802bbb0aaa', 'request_id': 'req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:12.455493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8647945f5f4f4a508d13e1802bbb0aaa', 'request_id': 'req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:12.462038 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8647945f5f4f4a508d13e1802bbb0aaa', 'request_id': 'req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:12.471402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:12.477152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] The token cache is being invalidate because project 8647945f5f4f4a508d13e1802bbb0aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:12:12.477675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:12.478775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-21ab1a22-1e4d-4df3-8df2-45bbb193b4b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:12.480380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1269/2547] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:12:12 2022] DELETE /identity/v3/projects/8647945f5f4f4a508d13e1802bbb0aaa => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:12.485245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85e8c6-f1ec-4c7b-8921-f7f5dcd155ce None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.485524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85e8c6-f1ec-4c7b-8921-f7f5dcd155ce None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.485959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85e8c6-f1ec-4c7b-8921-f7f5dcd155ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.506597 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ea85e8c6-f1ec-4c7b-8921-f7f5dcd155ce None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:12.507798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ea85e8c6-f1ec-4c7b-8921-f7f5dcd155ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:12.569652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1279/2548] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:12.577372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45096a79-9e72-4046-a114-e7ce93d6d72b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.577664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45096a79-9e72-4046-a114-e7ce93d6d72b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.577947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45096a79-9e72-4046-a114-e7ce93d6d72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.598780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-45096a79-9e72-4046-a114-e7ce93d6d72b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:12.600031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-45096a79-9e72-4046-a114-e7ce93d6d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:12.608204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1270/2549] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:12.613054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85e8c6-f1ec-4c7b-8921-f7f5dcd155ce None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.622574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85e8c6-f1ec-4c7b-8921-f7f5dcd155ce None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.627801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.628428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.628712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.628983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.629627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:12:12.629928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.631562 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.643145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1280/2550] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:12:12.647699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45096a79-9e72-4046-a114-e7ce93d6d72b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.657088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45096a79-9e72-4046-a114-e7ce93d6d72b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.661897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.662487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.662764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.663086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.663656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.665575 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.692854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1271/2551] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:12:12 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:12.696902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.706909 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11551735-2a85-4f19-97a8-452635c809b7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.762189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.762796 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.763078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.763355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.764054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.766017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.787229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1281/2552] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:12:12 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:12.791410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.802084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b2d55-b52b-4600-acf2-9c70c13fd691 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.806587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.807171 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.807468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.807723 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.808396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.809972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.820923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1272/2553] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:12.824861 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.834754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef7f8-b852-40ca-8092-42b3f1887ff6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.839297 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1d540-e821-4712-8868-e648b00026a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.839922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1d540-e821-4712-8868-e648b00026a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.840200 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1d540-e821-4712-8868-e648b00026a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.840470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1d540-e821-4712-8868-e648b00026a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.841104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1d540-e821-4712-8868-e648b00026a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.842668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1d540-e821-4712-8868-e648b00026a5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.853541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1282/2554] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:12:12.857643 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:12.868067 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3d5a5-97e5-414f-950c-e82b97eba701 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:12.872531 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:12.873158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.873430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.873697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] PATH_INFO: `/v3/projects/de8e3db3461e4c5792013c1c35366eeb/users/dbc2f8d05093490eb088be922b1d20bc/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.888222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8e3db3461e4c5792013c1c35366eeb, user_id=dbc2f8d05093490eb088be922b1d20bc, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:12.891543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:12.904405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1273/2555] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:12:12 2022] PUT /identity/v3/projects/de8e3db3461e4c5792013c1c35366eeb/users/dbc2f8d05093490eb088be922b1d20bc/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:12.915327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcf822-19ec-47e4-ace5-c4851a6836bf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:12.915632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcf822-19ec-47e4-ace5-c4851a6836bf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:12.915963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcf822-19ec-47e4-ace5-c4851a6836bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:12.944564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f1bcf822-19ec-47e4-ace5-c4851a6836bf None None] MFA Rules not processed for user `dbc2f8d05093490eb088be922b1d20bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:12.945927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f1bcf822-19ec-47e4-ace5-c4851a6836bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:12.959263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:13.014484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1283/2556] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:13.025712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:13.027379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:13.083442 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eb5c4-4990-47a2-a674-b1255ae2f61d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:13.087008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9dd1b-0b1a-4f51-a7a2-01a3236a8ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:13.087599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9dd1b-0b1a-4f51-a7a2-01a3236a8ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:13.087897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9dd1b-0b1a-4f51-a7a2-01a3236a8ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:13.088169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9dd1b-0b1a-4f51-a7a2-01a3236a8ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:13.096793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9dd1b-0b1a-4f51-a7a2-01a3236a8ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:13.098205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9dd1b-0b1a-4f51-a7a2-01a3236a8ed3 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:13.108183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1274/2557] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:16.104804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcf822-19ec-47e4-ace5-c4851a6836bf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:16.167015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcf822-19ec-47e4-ace5-c4851a6836bf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:16.171468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:16.172111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.172389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.172655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] PATH_INFO: `/v3/users/dbc2f8d05093490eb088be922b1d20bc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.174275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc2f8d05093490eb088be922b1d20bc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:16.176352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:16.221983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc2f8d05093490eb088be922b1d20bc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.233575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc2f8d05093490eb088be922b1d20bc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.241488 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc2f8d05093490eb088be922b1d20bc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.242095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc2f8d05093490eb088be922b1d20bc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.248904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc2f8d05093490eb088be922b1d20bc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.255251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1284/2558] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:12:16 2022] DELETE /identity/v3/users/dbc2f8d05093490eb088be922b1d20bc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:16.259696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.260208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.260321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.278745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:16.279950 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:16.336996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1275/2559] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:16.346515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:16.347982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:16.389186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:16.410154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12ab2d-458c-4228-9bf3-62a4e20f4314 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:16.413820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d358ca0-6a65-444a-96bc-4bd1484c8041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:16.414424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d358ca0-6a65-444a-96bc-4bd1484c8041 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.414701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d358ca0-6a65-444a-96bc-4bd1484c8041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.414971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d358ca0-6a65-444a-96bc-4bd1484c8041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.427175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d358ca0-6a65-444a-96bc-4bd1484c8041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:16.428684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d358ca0-6a65-444a-96bc-4bd1484c8041 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:16.437911 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1285/2560] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:16.704245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:16.705710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:16.718789 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b248b09-fc3c-4d7e-ad48-66f7e65679e6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:16.724368 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:16.724995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.725277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.726021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] PATH_INFO: `/v3/projects/de8e3db3461e4c5792013c1c35366eeb` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.726806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8e3db3461e4c5792013c1c35366eeb)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:16.728702 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:16.763748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:16.777543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8e3db3461e4c5792013c1c35366eeb', 'request_id': 'req-2c22aca0-4c35-4f80-a066-3c42246523dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.783867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8e3db3461e4c5792013c1c35366eeb', 'request_id': 'req-2c22aca0-4c35-4f80-a066-3c42246523dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.790392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8e3db3461e4c5792013c1c35366eeb', 'request_id': 'req-2c22aca0-4c35-4f80-a066-3c42246523dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.803061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] The token cache is being invalidate because project de8e3db3461e4c5792013c1c35366eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:12:16.803351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.804003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2c22aca0-4c35-4f80-a066-3c42246523dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:12:16.805580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1276/2561] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:12:16 2022] DELETE /identity/v3/projects/de8e3db3461e4c5792013c1c35366eeb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:12:16.810469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2bbee-1520-47e6-8332-8198b6e47b1e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.810758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2bbee-1520-47e6-8332-8198b6e47b1e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.811037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2bbee-1520-47e6-8332-8198b6e47b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.830207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-12e2bbee-1520-47e6-8332-8198b6e47b1e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:16.831409 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-12e2bbee-1520-47e6-8332-8198b6e47b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:16.889102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1286/2562] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:16.898822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f2f9c-24b6-4095-a8cc-35c82785bdc8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.899104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f2f9c-24b6-4095-a8cc-35c82785bdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.899389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f2f9c-24b6-4095-a8cc-35c82785bdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.919205 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-402f2f9c-24b6-4095-a8cc-35c82785bdc8 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:16.920425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-402f2f9c-24b6-4095-a8cc-35c82785bdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:16.928277 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1277/2563] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:16.932361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2bbee-1520-47e6-8332-8198b6e47b1e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:16.942147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2bbee-1520-47e6-8332-8198b6e47b1e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:16.946695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:16.947290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.947573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.947860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.948524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:12:16.948830 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:16.950435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:16.962544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1287/2564] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:12:16.966756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f2f9c-24b6-4095-a8cc-35c82785bdc8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:16.976514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f2f9c-24b6-4095-a8cc-35c82785bdc8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:16.981121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:16.981709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:16.981990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:16.982260 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:16.982893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:16.984854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:17.011499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1278/2565] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:12:16 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:12:17.015590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:17.027747 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4d26-5221-40a6-afe2-4e8f9abf9193 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:17.085289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:17.085943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:17.086228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:17.086640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:17.087364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:17.089349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:17.112515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1288/2566] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:12:17 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:12:17.116813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:17.127030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28826e2-476a-4413-b480-d7e8839a9afd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:17.131574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:17.132202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:17.132483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:17.132752 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:17.133389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:17.134996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:17.145351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1279/2567] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:17.149382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:17.159899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde163d-2d6e-4c86-8784-88e88c0e5115 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:17.164602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008c6de-c039-456d-8184-9006e436555c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:17.165193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008c6de-c039-456d-8184-9006e436555c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:17.165476 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008c6de-c039-456d-8184-9006e436555c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:17.165756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008c6de-c039-456d-8184-9006e436555c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:17.166396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008c6de-c039-456d-8184-9006e436555c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:17.168115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008c6de-c039-456d-8184-9006e436555c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:17.178866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1289/2568] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:12:17.182620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:17.192534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa6f59-fcc0-448b-a2f5-f6518f8db773 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:17.197105 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:17.197736 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:17.198015 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:17.198289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] PATH_INFO: `/v3/projects/c345dee22f114ebaa53969629a16d851/users/27621528118b4914b04bfa220116fe67/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:17.213012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c345dee22f114ebaa53969629a16d851, user_id=27621528118b4914b04bfa220116fe67, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:17.215647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:17.228611 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1280/2569] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:12:17 2022] PUT /identity/v3/projects/c345dee22f114ebaa53969629a16d851/users/27621528118b4914b04bfa220116fe67/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:12:17.238886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123683-d4f5-4248-814b-9fcf93b1364b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:17.239176 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123683-d4f5-4248-814b-9fcf93b1364b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:17.239497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123683-d4f5-4248-814b-9fcf93b1364b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:17.266834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8c123683-d4f5-4248-814b-9fcf93b1364b None None] MFA Rules not processed for user `27621528118b4914b04bfa220116fe67`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:12:17.268684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c123683-d4f5-4248-814b-9fcf93b1364b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:17.301639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:17.335310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1290/2570] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:12:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:12:17.347238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:12:17.349063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:12:17.409054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43157-edd1-4927-a1b6-3540878d5a11 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:12:17.412677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083954b-5d83-4468-bb2b-6db150b0fcf5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:12:17.413274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083954b-5d83-4468-bb2b-6db150b0fcf5 service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:12:17.413553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083954b-5d83-4468-bb2b-6db150b0fcf5 service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:12:17.413825 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083954b-5d83-4468-bb2b-6db150b0fcf5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:12:17.422016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083954b-5d83-4468-bb2b-6db150b0fcf5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:12:17.423439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083954b-5d83-4468-bb2b-6db150b0fcf5 service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:12:17.432380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1281/2571] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:12:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:12:27.332436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=12 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:12:27.428476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:05.553229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c123683-d4f5-4248-814b-9fcf93b1364b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:05.614689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c123683-d4f5-4248-814b-9fcf93b1364b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:05.619255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:05.619912 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:05.620197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:05.620473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] PATH_INFO: `/v3/users/27621528118b4914b04bfa220116fe67` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:05.622275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27621528118b4914b04bfa220116fe67)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:05.624375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:05.670907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27621528118b4914b04bfa220116fe67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:05.682377 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27621528118b4914b04bfa220116fe67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:05.690003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27621528118b4914b04bfa220116fe67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:05.690624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27621528118b4914b04bfa220116fe67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:05.697361 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27621528118b4914b04bfa220116fe67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:05.704004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1291/2572] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:13:05 2022] DELETE /identity/v3/users/27621528118b4914b04bfa220116fe67 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:05.708436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:05.708981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:05.708981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:05.727351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:05.728557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:05.789134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1282/2573] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:05.799293 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:05.800701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:05.877303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea724-4b79-4e88-9b3b-9ba8fad7ed74 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:05.880974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31ecf9-7ef2-4088-b138-84dc9e929573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:05.881566 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ecf9-7ef2-4088-b138-84dc9e929573 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:05.881842 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ecf9-7ef2-4088-b138-84dc9e929573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:05.882111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ecf9-7ef2-4088-b138-84dc9e929573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:05.891851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31ecf9-7ef2-4088-b138-84dc9e929573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:05.893672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31ecf9-7ef2-4088-b138-84dc9e929573 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:05.904291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1292/2574] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:06.184737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.186155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:06.197711 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28d77-4158-454f-9454-127a876dfe0d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.202960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.203583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.203886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.204165 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] PATH_INFO: `/v3/projects/c345dee22f114ebaa53969629a16d851` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.205303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c345dee22f114ebaa53969629a16d851)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.207190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.255074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c345dee22f114ebaa53969629a16d851', 'request_id': 'req-37a45912-221a-48eb-801b-2a08ca93bee2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:06.260591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c345dee22f114ebaa53969629a16d851', 'request_id': 'req-37a45912-221a-48eb-801b-2a08ca93bee2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:06.267408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c345dee22f114ebaa53969629a16d851', 'request_id': 'req-37a45912-221a-48eb-801b-2a08ca93bee2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:06.279468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] The token cache is being invalidate because project c345dee22f114ebaa53969629a16d851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:13:06.279755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:06.280404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37a45912-221a-48eb-801b-2a08ca93bee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:06.281887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1283/2575] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:13:06 2022] DELETE /identity/v3/projects/c345dee22f114ebaa53969629a16d851 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:06.286702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247fc67-0aaf-445d-b322-4f53dfcd8f94 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.286992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247fc67-0aaf-445d-b322-4f53dfcd8f94 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.287269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247fc67-0aaf-445d-b322-4f53dfcd8f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.294805 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:06.308329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6247fc67-0aaf-445d-b322-4f53dfcd8f94 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:06.310151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6247fc67-0aaf-445d-b322-4f53dfcd8f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:06.367483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1293/2576] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:06.376741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1365d3-134f-42b1-b848-809fe00f1263 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.377029 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1365d3-134f-42b1-b848-809fe00f1263 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.377303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1365d3-134f-42b1-b848-809fe00f1263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.398357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0b1365d3-134f-42b1-b848-809fe00f1263 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:06.399550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0b1365d3-134f-42b1-b848-809fe00f1263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:06.407911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1284/2577] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:06.412084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247fc67-0aaf-445d-b322-4f53dfcd8f94 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.422178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247fc67-0aaf-445d-b322-4f53dfcd8f94 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.426674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.427268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.427549 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.427823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.428498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:13:06.428800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.430378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.442700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1294/2578] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:13:06.446863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1365d3-134f-42b1-b848-809fe00f1263 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.456698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1365d3-134f-42b1-b848-809fe00f1263 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.461287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.461867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.462169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.462425 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.463046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.464976 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.494881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1285/2579] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:13:06 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:13:06.496047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.506864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b866846-9d35-4224-80e8-045be03737e8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.563452 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.564122 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.564340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.564608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.565224 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.567114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.588268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1295/2580] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:13:06 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:13:06.593245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.603634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b213919-1aa2-481c-a446-3ebb602b6359 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.608353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.608938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.609212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.609481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.610114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.611697 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.621482 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1286/2581] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:13:06.625280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.635168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5fb1-40cf-4de9-98ca-07705d366858 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.640327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b870c73-1e64-48f7-bbc9-6f93352775a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.640917 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b870c73-1e64-48f7-bbc9-6f93352775a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.641198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b870c73-1e64-48f7-bbc9-6f93352775a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.641471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b870c73-1e64-48f7-bbc9-6f93352775a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.642105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b870c73-1e64-48f7-bbc9-6f93352775a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.643683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b870c73-1e64-48f7-bbc9-6f93352775a7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.655329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1296/2582] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:13:06.659631 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.669480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab7223-ca1a-4aa6-a1ce-c5e8d4899efb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.673978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.674634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.674871 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.675169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] PATH_INFO: `/v3/projects/c93b5efcd24c4260a7b27b2b1444d5af/users/d027b0f0928846d39696b4e9f729182c/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.689405 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c93b5efcd24c4260a7b27b2b1444d5af, user_id=d027b0f0928846d39696b4e9f729182c, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.692032 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.704970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1287/2583] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:13:06 2022] PUT /identity/v3/projects/c93b5efcd24c4260a7b27b2b1444d5af/users/d027b0f0928846d39696b4e9f729182c/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:06.716063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee756b12-67b1-4583-bdaf-3d85f9946ef1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.716539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee756b12-67b1-4583-bdaf-3d85f9946ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.716823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee756b12-67b1-4583-bdaf-3d85f9946ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.744425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ee756b12-67b1-4583-bdaf-3d85f9946ef1 None None] MFA Rules not processed for user `d027b0f0928846d39696b4e9f729182c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:06.745625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee756b12-67b1-4583-bdaf-3d85f9946ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:06.804902 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:06.810904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1297/2584] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:06.822484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:06.824285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:06.847444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:06.883765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9c1a4-5cfe-4099-a549-6a1aab9fdd50 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:06.888299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:06.888911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:06.889190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:06.889459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:06.897889 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:06.899320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:06.908321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1288/2585] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:16.812049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:16.904427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=17 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:18.016684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de63421-0bcc-4556-a308-2c080bc59a06 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:18.017192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de63421-0bcc-4556-a308-2c080bc59a06 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:18.017305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de63421-0bcc-4556-a308-2c080bc59a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:18.047529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6de63421-0bcc-4556-a308-2c080bc59a06 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:18.048820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6de63421-0bcc-4556-a308-2c080bc59a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:18.116390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1298/2586] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:13:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:18.159320 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:18.160840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:18.240223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab42399-7c03-4ef3-9a8d-1726d7297208 service glance] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:18.245154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80082b9-b5e2-4f8e-ab10-e3bad41cf976 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:18.245757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80082b9-b5e2-4f8e-ab10-e3bad41cf976 service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:18.246035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80082b9-b5e2-4f8e-ab10-e3bad41cf976 service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:18.246307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80082b9-b5e2-4f8e-ab10-e3bad41cf976 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:18.254989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80082b9-b5e2-4f8e-ab10-e3bad41cf976 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:18.256668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80082b9-b5e2-4f8e-ab10-e3bad41cf976 service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:18.265549 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1289/2587] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:13:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:13:28.109194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=32 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:28.261529 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=32 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:37.830948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac073-c87b-44c4-9395-f94fc0579e61 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:37.831563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac073-c87b-44c4-9395-f94fc0579e61 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:37.831563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac073-c87b-44c4-9395-f94fc0579e61 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:37.833072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1299/2588] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:13:37.838497 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d7963-9826-4b1c-8935-ec5510ef455e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:37.839010 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d7963-9826-4b1c-8935-ec5510ef455e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:37.839157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d7963-9826-4b1c-8935-ec5510ef455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:37.872913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5f6d7963-9826-4b1c-8935-ec5510ef455e None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:37.874911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5f6d7963-9826-4b1c-8935-ec5510ef455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:37.959428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1290/2589] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:13:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:13:37.971407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac073-c87b-44c4-9395-f94fc0579e61 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:37.982460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac073-c87b-44c4-9395-f94fc0579e61 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:37.986100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e68419-bb54-4cbc-9ed4-bbdf2f5dc36b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f85882ffc0447baaf3e00554f6b4ed1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:37.986693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e68419-bb54-4cbc-9ed4-bbdf2f5dc36b service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:37.986973 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e68419-bb54-4cbc-9ed4-bbdf2f5dc36b service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:37.987242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e68419-bb54-4cbc-9ed4-bbdf2f5dc36b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:37.997534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e68419-bb54-4cbc-9ed4-bbdf2f5dc36b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:37.998978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e68419-bb54-4cbc-9ed4-bbdf2f5dc36b service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:38.009624 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1300/2590] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:38.079267 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f27d4-fe0e-4a4a-9dcf-c80881d2cd7a None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:38.079555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f27d4-fe0e-4a4a-9dcf-c80881d2cd7a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:38.079829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f27d4-fe0e-4a4a-9dcf-c80881d2cd7a None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:38.081212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1291/2591] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:13:38.085991 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3c59-4595-4830-a4df-e2b489c22c07 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:38.086281 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3c59-4595-4830-a4df-e2b489c22c07 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:38.086586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3c59-4595-4830-a4df-e2b489c22c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:38.106616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9f8d3c59-4595-4830-a4df-e2b489c22c07 None None] MFA Rules not processed for user `b5c892640b3e409b9a99b4ca1d34a964`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:38.107822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f8d3c59-4595-4830-a4df-e2b489c22c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:38.160268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1301/2592] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:13:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:38.172322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f27d4-fe0e-4a4a-9dcf-c80881d2cd7a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:38.173998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-199f27d4-fe0e-4a4a-9dcf-c80881d2cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:38.186903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f27d4-fe0e-4a4a-9dcf-c80881d2cd7a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:38.190314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bba2f-ebe5-47fc-bd33-01c489a79b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:38.190890 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bba2f-ebe5-47fc-bd33-01c489a79b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:38.191163 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bba2f-ebe5-47fc-bd33-01c489a79b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:38.191432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bba2f-ebe5-47fc-bd33-01c489a79b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:38.198156 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bba2f-ebe5-47fc-bd33-01c489a79b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:38.199560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bba2f-ebe5-47fc-bd33-01c489a79b5a service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:38.208445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1292/2593] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:38.293479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97944a1a-23fe-4c21-9ab6-e941086253f3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:38.293772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97944a1a-23fe-4c21-9ab6-e941086253f3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:38.294059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97944a1a-23fe-4c21-9ab6-e941086253f3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:38.295415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1302/2594] 10.222.0.52 () {54 vars in 921 bytes} [Thu Oct 6 11:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 11:13:38.300724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1efd7-4680-4ff2-a96d-cb02a3b5d2f9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:38.301008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1efd7-4680-4ff2-a96d-cb02a3b5d2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:38.301281 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1efd7-4680-4ff2-a96d-cb02a3b5d2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:38.321305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-48e1efd7-4680-4ff2-a96d-cb02a3b5d2f9 None None] MFA Rules not processed for user `91413e6647284487bae08bb7eb02da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:38.322536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-48e1efd7-4680-4ff2-a96d-cb02a3b5d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:38.332680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1293/2595] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Oct 6 11:13:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:38.343248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97944a1a-23fe-4c21-9ab6-e941086253f3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:38.354210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97944a1a-23fe-4c21-9ab6-e941086253f3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:38.357823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:38.358576 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:38.358869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:38.359147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:38.369111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:38.370524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:38.382311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1303/2596] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:40.228872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd068b-8f09-4783-b7ac-cb0b0250c336 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:40.229417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd068b-8f09-4783-b7ac-cb0b0250c336 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:40.229417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd068b-8f09-4783-b7ac-cb0b0250c336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:40.251472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-accd068b-8f09-4783-b7ac-cb0b0250c336 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:40.252769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-accd068b-8f09-4783-b7ac-cb0b0250c336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:40.260220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1294/2597] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Oct 6 11:13:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:40.273082 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:40.284903 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eb383-a01a-45a9-8037-c19b2d7a69af service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:40.290216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81a524-4937-4761-b1ba-f504289636d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:40.291002 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81a524-4937-4761-b1ba-f504289636d7 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:40.291290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81a524-4937-4761-b1ba-f504289636d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:40.291567 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81a524-4937-4761-b1ba-f504289636d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:40.301242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81a524-4937-4761-b1ba-f504289636d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:40.302875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81a524-4937-4761-b1ba-f504289636d7 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:40.312648 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1304/2598] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:13:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:13:43.270386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd068b-8f09-4783-b7ac-cb0b0250c336 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:43.354235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd068b-8f09-4783-b7ac-cb0b0250c336 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:43.360134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:43.360774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:43.361054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:43.361337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] PATH_INFO: `/v3/users/d027b0f0928846d39696b4e9f729182c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:43.362493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=d027b0f0928846d39696b4e9f729182c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:43.364477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:43.416882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd027b0f0928846d39696b4e9f729182c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:43.418218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:43.439876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd027b0f0928846d39696b4e9f729182c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:43.439876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd027b0f0928846d39696b4e9f729182c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:43.447229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd027b0f0928846d39696b4e9f729182c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:43.448133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd027b0f0928846d39696b4e9f729182c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:43.461287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1295/2599] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:13:43 2022] DELETE /identity/v3/users/d027b0f0928846d39696b4e9f729182c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:43.465945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:43.466493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:43.466493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:43.488236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:43.489471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:43.572738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1305/2600] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:43.583228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:43.584716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:43.644913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a3fc-2732-4ca2-a1ac-70b3a3e1b2be admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:43.648862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71a8eb-cb38-488e-9634-58f3c5aa192b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:43.649472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71a8eb-cb38-488e-9634-58f3c5aa192b service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:43.649748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71a8eb-cb38-488e-9634-58f3c5aa192b service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:43.650016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71a8eb-cb38-488e-9634-58f3c5aa192b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:43.660796 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71a8eb-cb38-488e-9634-58f3c5aa192b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:43.662237 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71a8eb-cb38-488e-9634-58f3c5aa192b service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:43.674191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1296/2601] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:43.990824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:43.992645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:44.042604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ff139-4204-44ee-a9e2-8246aa7f502f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:44.059511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:44.062751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.063062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.063342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] PATH_INFO: `/v3/projects/c93b5efcd24c4260a7b27b2b1444d5af` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.064700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c93b5efcd24c4260a7b27b2b1444d5af)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:44.066726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:44.134811 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c93b5efcd24c4260a7b27b2b1444d5af', 'request_id': 'req-2c95cf65-ca0d-4110-96a1-14b94e26b49b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:44.141233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c93b5efcd24c4260a7b27b2b1444d5af', 'request_id': 'req-2c95cf65-ca0d-4110-96a1-14b94e26b49b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:44.155819 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c93b5efcd24c4260a7b27b2b1444d5af', 'request_id': 'req-2c95cf65-ca0d-4110-96a1-14b94e26b49b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:44.160804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:44.177846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] The token cache is being invalidate because project c93b5efcd24c4260a7b27b2b1444d5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:13:44.178213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:44.179007 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c95cf65-ca0d-4110-96a1-14b94e26b49b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:44.181951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1306/2602] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:13:43 2022] DELETE /identity/v3/projects/c93b5efcd24c4260a7b27b2b1444d5af => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:44.186792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9f03f-6122-4a0a-bba5-f83a3eb9325f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.187054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9f03f-6122-4a0a-bba5-f83a3eb9325f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.187360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9f03f-6122-4a0a-bba5-f83a3eb9325f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.209194 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2ed9f03f-6122-4a0a-bba5-f83a3eb9325f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:44.210418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2ed9f03f-6122-4a0a-bba5-f83a3eb9325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:44.276227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1297/2603] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:44.345157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c7220-38f6-4e88-94fb-5579e52dc070 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.345444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c7220-38f6-4e88-94fb-5579e52dc070 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.345739 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c7220-38f6-4e88-94fb-5579e52dc070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.366933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c84c7220-38f6-4e88-94fb-5579e52dc070 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:44.368182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c84c7220-38f6-4e88-94fb-5579e52dc070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:44.376419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1307/2604] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:13:44.381366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9f03f-6122-4a0a-bba5-f83a3eb9325f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:44.392016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9f03f-6122-4a0a-bba5-f83a3eb9325f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:44.396590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:44.397574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.397739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.398021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.398666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:13:44.398972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:44.400636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:44.414204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1298/2605] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:13:44.418875 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c7220-38f6-4e88-94fb-5579e52dc070 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:44.429285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c7220-38f6-4e88-94fb-5579e52dc070 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:44.434039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:44.434636 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.434918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.435193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.435857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:44.437969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:44.470594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1308/2606] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:13:44 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:13:44.474419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:44.485891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df979-0fc3-4a36-8d1b-ec482f4a8357 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:44.537517 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:44.545232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:44.546459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.546817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.547098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.547768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:44.551240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:44.580061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1299/2607] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:13:44 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:13:44.581477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:44.595191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b14ed7-e664-4e29-9665-c20e8b65e340 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:44.600251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:44.600945 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.601405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.601776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.602514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:44.604198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:44.615590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1309/2608] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:13:44.619712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:44.630316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d050a-1bbc-42c4-be41-c5790d43c683 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:44.635008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27cad4-71b9-45c2-ae5d-84a9565ac680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:44.635545 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27cad4-71b9-45c2-ae5d-84a9565ac680 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.635816 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27cad4-71b9-45c2-ae5d-84a9565ac680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.636144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27cad4-71b9-45c2-ae5d-84a9565ac680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.636807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27cad4-71b9-45c2-ae5d-84a9565ac680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:44.638403 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27cad4-71b9-45c2-ae5d-84a9565ac680 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:44.659238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1300/2609] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:13:44.663510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:44.675212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b5cd2-4e7a-49b9-8424-ba4fbd00ae2c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:44.679809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:44.680484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.680764 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.681036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] PATH_INFO: `/v3/projects/a3d48e16a5eb4d1daac05ac839d9738d/users/eed10211d3f845a8948955ec00c9413c/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.824854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d48e16a5eb4d1daac05ac839d9738d, user_id=eed10211d3f845a8948955ec00c9413c, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:44.827812 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:44.842346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1310/2610] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:13:44 2022] PUT /identity/v3/projects/a3d48e16a5eb4d1daac05ac839d9738d/users/eed10211d3f845a8948955ec00c9413c/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:44.852235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5730-2a5e-4ff0-a72a-250586687b9e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:44.852834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5730-2a5e-4ff0-a72a-250586687b9e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:44.853155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5730-2a5e-4ff0-a72a-250586687b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:44.891432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e3dd5730-2a5e-4ff0-a72a-250586687b9e None None] MFA Rules not processed for user `eed10211d3f845a8948955ec00c9413c`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:44.892901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3dd5730-2a5e-4ff0-a72a-250586687b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:44.962404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1301/2611] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:44.973557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:44.975340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:45.057071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be291627-e2e4-4b2b-a997-bca7e67fcda8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:45.060843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d850d-0574-4888-aadd-2b009dcbe0df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:45.061516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d850d-0574-4888-aadd-2b009dcbe0df service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:45.062308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d850d-0574-4888-aadd-2b009dcbe0df service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:45.062797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d850d-0574-4888-aadd-2b009dcbe0df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:45.072719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d850d-0574-4888-aadd-2b009dcbe0df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:45.074147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d850d-0574-4888-aadd-2b009dcbe0df service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:45.085893 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1311/2612] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:45.113060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd5730-2a5e-4ff0-a72a-250586687b9e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:45.181365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd5730-2a5e-4ff0-a72a-250586687b9e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:45.185942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:45.186602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:45.187065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:45.187570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] PATH_INFO: `/v3/users/eed10211d3f845a8948955ec00c9413c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:45.188808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed10211d3f845a8948955ec00c9413c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:45.190885 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:45.252799 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed10211d3f845a8948955ec00c9413c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.267164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed10211d3f845a8948955ec00c9413c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.268839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed10211d3f845a8948955ec00c9413c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.271668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:45.283599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed10211d3f845a8948955ec00c9413c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.284619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed10211d3f845a8948955ec00c9413c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.303516 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1302/2613] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:13:45 2022] DELETE /identity/v3/users/eed10211d3f845a8948955ec00c9413c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:45.308030 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:45.308315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:45.308608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:45.328920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:45.330138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:45.409687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1312/2614] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:45.416492 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:45.417910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:45.480073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9fad8-1005-4151-ae92-851aa27d52b1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:45.483892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da4bac-7025-41c4-97f3-712b9067cd53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:45.484503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4bac-7025-41c4-97f3-712b9067cd53 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:45.484780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4bac-7025-41c4-97f3-712b9067cd53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:45.485051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4bac-7025-41c4-97f3-712b9067cd53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:45.495053 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da4bac-7025-41c4-97f3-712b9067cd53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:45.496522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da4bac-7025-41c4-97f3-712b9067cd53 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:45.506526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1303/2615] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:45.790504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:45.792608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:45.818708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79bf5-714e-4016-a779-bafea4357d07 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:45.824620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:45.825405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:45.825748 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:45.826106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] PATH_INFO: `/v3/projects/a3d48e16a5eb4d1daac05ac839d9738d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:45.828107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d48e16a5eb4d1daac05ac839d9738d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:45.830497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:45.839474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:45.895680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d48e16a5eb4d1daac05ac839d9738d', 'request_id': 'req-9905a36f-9a9e-4e2d-8441-72895fb887fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.901659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d48e16a5eb4d1daac05ac839d9738d', 'request_id': 'req-9905a36f-9a9e-4e2d-8441-72895fb887fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.908690 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d48e16a5eb4d1daac05ac839d9738d', 'request_id': 'req-9905a36f-9a9e-4e2d-8441-72895fb887fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.922757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] The token cache is being invalidate because project a3d48e16a5eb4d1daac05ac839d9738d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:13:45.923048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.923681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9905a36f-9a9e-4e2d-8441-72895fb887fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:13:45.925454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1313/2616] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:13:45 2022] DELETE /identity/v3/projects/a3d48e16a5eb4d1daac05ac839d9738d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:45.930483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fad366-7b8f-4385-8393-ab2e26fab13b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:45.930771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fad366-7b8f-4385-8393-ab2e26fab13b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:45.931037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fad366-7b8f-4385-8393-ab2e26fab13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:45.954748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-19fad366-7b8f-4385-8393-ab2e26fab13b None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:45.955989 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-19fad366-7b8f-4385-8393-ab2e26fab13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:46.020577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1304/2617] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:46.028889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30059c0c-78e8-4650-bfba-d06d531cee85 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.029291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30059c0c-78e8-4650-bfba-d06d531cee85 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.029570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30059c0c-78e8-4650-bfba-d06d531cee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.051108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-30059c0c-78e8-4650-bfba-d06d531cee85 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:46.052463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-30059c0c-78e8-4650-bfba-d06d531cee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:46.060841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1314/2618] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:46.065734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fad366-7b8f-4385-8393-ab2e26fab13b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:46.075727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fad366-7b8f-4385-8393-ab2e26fab13b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:46.081041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:46.081641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.081922 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.082197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.082847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:13:46.083150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:46.084851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:46.097363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1305/2619] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:13:46.102051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30059c0c-78e8-4650-bfba-d06d531cee85 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:46.113089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30059c0c-78e8-4650-bfba-d06d531cee85 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:46.118634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:46.119298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.119716 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.120044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.120749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:46.122803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:46.153342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1315/2620] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:13:46 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:13:46.157661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:46.167968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceedfaa-9065-40c1-80c6-1b9e9a3470ec admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:46.239341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:46.240147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.240448 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.240773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.241422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:46.243401 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:46.266195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1306/2621] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:13:46 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:13:46.270374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:46.281416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023d671-d47a-4584-b17f-5ad76fbe8faa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:46.286207 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:46.286813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.287089 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.287356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.288041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:46.289617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:46.301240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1316/2622] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:13:46.305001 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:46.310159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:46.318741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8984a-e8d6-466b-aebb-a69f6801474d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:46.324953 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498817e-d10f-4492-8c9b-98b92ca52da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:46.325553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498817e-d10f-4492-8c9b-98b92ca52da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.325828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498817e-d10f-4492-8c9b-98b92ca52da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.326097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498817e-d10f-4492-8c9b-98b92ca52da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.326734 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498817e-d10f-4492-8c9b-98b92ca52da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:46.328881 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498817e-d10f-4492-8c9b-98b92ca52da4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:46.339960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1307/2623] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:13:46.344508 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:46.354720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e16cd-66a1-403f-8f54-e94f212caa15 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:46.359958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:46.360822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.361268 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.361738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] PATH_INFO: `/v3/projects/9376d8470a80416cb8e57eb650884883/users/e00353b2edab47e79578de680e4c4d1c/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.378131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9376d8470a80416cb8e57eb650884883, user_id=e00353b2edab47e79578de680e4c4d1c, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:46.391348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:46.415184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1317/2624] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:13:46 2022] PUT /identity/v3/projects/9376d8470a80416cb8e57eb650884883/users/e00353b2edab47e79578de680e4c4d1c/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:13:46.420429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68a381-6205-4abf-b1da-4890531a8a2e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.420832 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68a381-6205-4abf-b1da-4890531a8a2e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.421162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68a381-6205-4abf-b1da-4890531a8a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.443229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5f68a381-6205-4abf-b1da-4890531a8a2e None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:46.444737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5f68a381-6205-4abf-b1da-4890531a8a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:46.519420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1308/2625] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:13:46.529771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:46.532201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:46.604422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ef28c-b439-4a6d-9e34-32fe97c36949 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:46.609016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:46.609642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:46.609962 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:46.610193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:46.620404 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:46.621901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:46.634238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1318/2626] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:51.875462 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477892c6-a18c-4167-8b9a-bf80dda22914 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:51.876131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477892c6-a18c-4167-8b9a-bf80dda22914 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:51.876131 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477892c6-a18c-4167-8b9a-bf80dda22914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:51.906933 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-477892c6-a18c-4167-8b9a-bf80dda22914 None None] MFA Rules not processed for user `e00353b2edab47e79578de680e4c4d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:51.908269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-477892c6-a18c-4167-8b9a-bf80dda22914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:51.976910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1309/2627] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:51.981440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:51.991510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2b60f-3c67-4afa-91f6-a7c4592c8a9c service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:51.996320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:51.996925 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:51.997202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:51.997469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:51.998107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:52.000770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:52.039633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1319/2628] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:13:51 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:13:52.044113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477892c6-a18c-4167-8b9a-bf80dda22914 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:52.061762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477892c6-a18c-4167-8b9a-bf80dda22914 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:52.133942 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:52.134610 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:52.134894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:52.135167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:52.135803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:52.137773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:52.163069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1310/2629] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:13:52 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:13:52.167131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:52.180010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7d519-a206-4b43-9367-8461b7465c94 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:52.188263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:52.188828 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:52.189111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:52.189386 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:52.190094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:52.191713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:52.204753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1320/2630] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:13:52.209172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:52.220030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf058db-525e-4b37-ba99-21c5526a69e8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:52.225410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:52.226086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:52.226685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:52.226685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] PATH_INFO: `/v3/projects/7b6d1cbcfb8f4dffa56b44651c1316fa/users/17da95f02da34b0896ac24fbb3276359/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:52.236900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:52.245390 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6d1cbcfb8f4dffa56b44651c1316fa, user_id=17da95f02da34b0896ac24fbb3276359, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:52.256129 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:52.263543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1311/2631] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:13:52 2022] PUT /identity/v3/projects/7b6d1cbcfb8f4dffa56b44651c1316fa/users/17da95f02da34b0896ac24fbb3276359/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:52.268431 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:52.273986 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:13:52.280217 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc766ddd-75d6-4767-a48a-334e6bda72fd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:52.347232 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afce7a-6ab8-4494-95ef-915eef7d22e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:52.348032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afce7a-6ab8-4494-95ef-915eef7d22e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:52.348376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afce7a-6ab8-4494-95ef-915eef7d22e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:52.348773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afce7a-6ab8-4494-95ef-915eef7d22e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:52.349509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afce7a-6ab8-4494-95ef-915eef7d22e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:52.351096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afce7a-6ab8-4494-95ef-915eef7d22e2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:52.361785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1321/2632] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:13:52.366114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:52.377071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45fef-dc54-4191-85c9-f234b1fdbd2d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:52.381700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:52.382488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:52.382769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:52.383042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] PATH_INFO: `/v3/projects/7b6d1cbcfb8f4dffa56b44651c1316fa/users/17da95f02da34b0896ac24fbb3276359/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:52.385048 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6d1cbcfb8f4dffa56b44651c1316fa, user_id=17da95f02da34b0896ac24fbb3276359, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:52.387698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:52.402143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1312/2633] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:13:52 2022] PUT /identity/v3/projects/7b6d1cbcfb8f4dffa56b44651c1316fa/users/17da95f02da34b0896ac24fbb3276359/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:13:56.996956 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd852-8ce8-49ec-a1aa-28383f003aa5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:56.997509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd852-8ce8-49ec-a1aa-28383f003aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:56.997509 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd852-8ce8-49ec-a1aa-28383f003aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:57.026481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eb1cd852-8ce8-49ec-a1aa-28383f003aa5 None None] MFA Rules not processed for user `17da95f02da34b0896ac24fbb3276359`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:57.027798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eb1cd852-8ce8-49ec-a1aa-28383f003aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:57.103319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1322/2634] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:57.118755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:57.122626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:57.223687 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7499e26-bd28-4780-b49b-1c6790b45d7b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:57.230544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:57.231220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:57.231502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:57.231773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:57.298486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:57.300266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:57.315858 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1313/2635] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:13:58.461530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464d90c-e66d-47f9-8bc5-9af767985dc3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:58.462040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464d90c-e66d-47f9-8bc5-9af767985dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:58.462162 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464d90c-e66d-47f9-8bc5-9af767985dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:58.483752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3464d90c-e66d-47f9-8bc5-9af767985dc3 None None] MFA Rules not processed for user `e00353b2edab47e79578de680e4c4d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:13:58.485265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3464d90c-e66d-47f9-8bc5-9af767985dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:58.496220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1323/2636] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:13:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:13:59.347098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:13:59.348540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:59.411944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32583b-dcca-4002-85f7-086b76f1c342 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:13:59.415642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49b4c-b54c-4644-a979-62b1702119ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:13:59.416223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49b4c-b54c-4644-a979-62b1702119ec service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:13:59.416501 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49b4c-b54c-4644-a979-62b1702119ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:13:59.416774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49b4c-b54c-4644-a979-62b1702119ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:13:59.418472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6ad49b4c-b54c-4644-a979-62b1702119ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:13:59.435319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49b4c-b54c-4644-a979-62b1702119ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:13:59.438428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49b4c-b54c-4644-a979-62b1702119ec service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:13:59.457080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1314/2637] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:13:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:00.803780 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464d90c-e66d-47f9-8bc5-9af767985dc3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:00.814777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464d90c-e66d-47f9-8bc5-9af767985dc3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:00.818732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:00.819398 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:00.819794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:00.820277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:00.822651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:00.838257 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:00.839679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:00.851588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1324/2638] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:01.045970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f8f9a-4ff5-4ae6-8ea2-0c6d8432700c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:01.046775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f8f9a-4ff5-4ae6-8ea2-0c6d8432700c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:01.046775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f8f9a-4ff5-4ae6-8ea2-0c6d8432700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:01.066593 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-911f8f9a-4ff5-4ae6-8ea2-0c6d8432700c None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:01.067727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-911f8f9a-4ff5-4ae6-8ea2-0c6d8432700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:01.076688 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:01.132479 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1315/2639] 10.222.0.52 () {58 vars in 967 bytes} [Thu Oct 6 11:14:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:14:01.151896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:01.153295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:01.239301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd76749-63f6-465c-836c-cb816e1e52a6 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:01.244225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:01.244829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:01.245137 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:01.245406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:01.251440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:01.255573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:01.257814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:01.268681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1325/2640] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:14:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 06 11:14:11.125763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=19 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:11.264957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:21.534884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f8f9a-4ff5-4ae6-8ea2-0c6d8432700c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:21.543994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f8f9a-4ff5-4ae6-8ea2-0c6d8432700c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:21.547469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:21.548082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:21.548359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:21.548628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:21.550322 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:21.560460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:21.561873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:21.570460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1316/2641] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:31.566167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:39.968110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:39.977891 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4715d-fa4c-4002-ad76-19c773f0ed4e service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:39.981439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:39.982023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:39.982301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:39.982571 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:39.984375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:40.046590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:40.048039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:40.058308 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1326/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:42.282640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:42.292317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a62fd-3430-4d1e-b6ec-f3ebc571a209 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:42.296000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:42.296595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:42.296876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:42.297147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:42.307457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:42.308943 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:42.318586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1317/2643] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:43.652822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:43.654417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:43.669056 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3357-12ac-4cd8-ae0a-59600d8adc96 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:43.674402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:43.675033 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:43.675483 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:43.676436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] PATH_INFO: `/v3/users/1bb1115590ac45fdbcdc6d3f52aa6921` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:43.694683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb1115590ac45fdbcdc6d3f52aa6921)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:43.701320 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:43.751899 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb1115590ac45fdbcdc6d3f52aa6921'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:43.769055 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb1115590ac45fdbcdc6d3f52aa6921'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:43.778319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb1115590ac45fdbcdc6d3f52aa6921'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:43.779019 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb1115590ac45fdbcdc6d3f52aa6921'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:43.787926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b1b3928-f8a4-47e0-a9fa-1d145698a90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb1115590ac45fdbcdc6d3f52aa6921'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:43.795080 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1327/2644] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:14:43 2022] DELETE /identity/v3/users/1bb1115590ac45fdbcdc6d3f52aa6921 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:43.991028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.005528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfa6b9-9e89-4314-b2a4-2e3d890f30b8 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.071518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.072852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.073138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.073418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] PATH_INFO: `/v3/projects/8c9bfd4947d045cf831f1a60bf91f389` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.085224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9bfd4947d045cf831f1a60bf91f389)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.087162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.137088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9bfd4947d045cf831f1a60bf91f389', 'request_id': 'req-82cd40ed-6c34-4ede-8466-df35d2dd2c62'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:44.142412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9bfd4947d045cf831f1a60bf91f389', 'request_id': 'req-82cd40ed-6c34-4ede-8466-df35d2dd2c62'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:44.150533 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9bfd4947d045cf831f1a60bf91f389', 'request_id': 'req-82cd40ed-6c34-4ede-8466-df35d2dd2c62'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:44.164851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] The token cache is being invalidate because project 8c9bfd4947d045cf831f1a60bf91f389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:14:44.165299 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:44.166093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-82cd40ed-6c34-4ede-8466-df35d2dd2c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:44.168188 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1318/2645] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:14:43 2022] DELETE /identity/v3/projects/8c9bfd4947d045cf831f1a60bf91f389 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:44.172958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9589f-818b-41f1-a041-7731c302b7da None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.173245 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9589f-818b-41f1-a041-7731c302b7da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.173519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9589f-818b-41f1-a041-7731c302b7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.193543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fda9589f-818b-41f1-a041-7731c302b7da None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:44.194686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fda9589f-818b-41f1-a041-7731c302b7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:44.245101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:44.259338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1328/2646] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:14:44.267314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9cfd-ce23-40f3-8ed6-273d8f01b8c2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.267602 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9cfd-ce23-40f3-8ed6-273d8f01b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.267905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9cfd-ce23-40f3-8ed6-273d8f01b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.288251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b6ab9cfd-ce23-40f3-8ed6-273d8f01b8c2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:44.289456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b6ab9cfd-ce23-40f3-8ed6-273d8f01b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:44.297342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1319/2647] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:14:44.302214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9589f-818b-41f1-a041-7731c302b7da None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.311921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9589f-818b-41f1-a041-7731c302b7da None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.317188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.317787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.318062 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.318337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.319024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:14:44.319319 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.320918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.333243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1329/2648] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:14:44.337450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab9cfd-ce23-40f3-8ed6-273d8f01b8c2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.346815 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab9cfd-ce23-40f3-8ed6-273d8f01b8c2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.351469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.352088 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.352363 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.352632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.353263 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.355172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.399880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1320/2649] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:14:44 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:14:44.404684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.414926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba2ca-9ee0-4071-a257-8a6ba276f50f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.473541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.473541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.473881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.474049 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.474670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.476984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.508139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1330/2650] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:14:44 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:14:44.511204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.523173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbe2ee-6bbb-4aa7-8d61-2f19625a3941 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.528340 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.528948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.529229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.529506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.530154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.532530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.545203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1321/2651] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:14:44.548807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.560670 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91035e-1ba1-4f43-901b-efc6635682fe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.565725 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee80aa1-9f04-4e15-b9ae-476531ddd7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.567291 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80aa1-9f04-4e15-b9ae-476531ddd7fd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.567640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80aa1-9f04-4e15-b9ae-476531ddd7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.567957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80aa1-9f04-4e15-b9ae-476531ddd7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.568614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee80aa1-9f04-4e15-b9ae-476531ddd7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.570316 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee80aa1-9f04-4e15-b9ae-476531ddd7fd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.584071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1331/2652] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:14:44.588380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.598971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100e4f7-08bb-4ef7-8f02-7f710104fdc3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.603556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.604229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.604508 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.604771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] PATH_INFO: `/v3/projects/946d081e3c9143659e7f805a368fd6eb/users/075061bd2a174dc5823d04f292c9de7b/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.620330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946d081e3c9143659e7f805a368fd6eb, user_id=075061bd2a174dc5823d04f292c9de7b, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.623071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.626346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:44.638333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1322/2653] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:14:44 2022] PUT /identity/v3/projects/946d081e3c9143659e7f805a368fd6eb/users/075061bd2a174dc5823d04f292c9de7b/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:44.651706 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01cd22-d2e3-43ac-8cd6-b69805a0c318 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.652036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01cd22-d2e3-43ac-8cd6-b69805a0c318 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.652460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01cd22-d2e3-43ac-8cd6-b69805a0c318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.687184 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ac01cd22-d2e3-43ac-8cd6-b69805a0c318 None None] MFA Rules not processed for user `075061bd2a174dc5823d04f292c9de7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:44.688240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ac01cd22-d2e3-43ac-8cd6-b69805a0c318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:44.761608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1332/2654] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:14:44.773272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:44.775064 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:44.856962 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee1e48-6442-464e-947b-f6422ae58bf8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:44.863622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef494d-6033-4dd0-8983-aaa7f9f8de8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:44.864328 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef494d-6033-4dd0-8983-aaa7f9f8de8b service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:44.864622 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef494d-6033-4dd0-8983-aaa7f9f8de8b service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:44.865051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef494d-6033-4dd0-8983-aaa7f9f8de8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:44.873564 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef494d-6033-4dd0-8983-aaa7f9f8de8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:44.875258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef494d-6033-4dd0-8983-aaa7f9f8de8b service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:44.885591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1323/2655] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:45.142192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01cd22-d2e3-43ac-8cd6-b69805a0c318 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:45.198818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:45.212744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01cd22-d2e3-43ac-8cd6-b69805a0c318 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:45.217838 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:45.218885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:45.218885 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:45.219124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] PATH_INFO: `/v3/users/075061bd2a174dc5823d04f292c9de7b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:45.220596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=075061bd2a174dc5823d04f292c9de7b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:45.222708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:45.272492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '075061bd2a174dc5823d04f292c9de7b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.285264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '075061bd2a174dc5823d04f292c9de7b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.294198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '075061bd2a174dc5823d04f292c9de7b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.294862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '075061bd2a174dc5823d04f292c9de7b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.302817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '075061bd2a174dc5823d04f292c9de7b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.309392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1333/2656] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:14:45 2022] DELETE /identity/v3/users/075061bd2a174dc5823d04f292c9de7b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:45.313589 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:45.314092 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:45.314194 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:45.333853 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:45.335059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:45.404596 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1324/2657] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:14:45.414896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:45.416341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:45.474882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff44f5-1b60-4d49-be05-eedea753e76e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:45.478553 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c4c88-1164-4ff2-b93f-f6e7ba8c4834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:45.479172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4c88-1164-4ff2-b93f-f6e7ba8c4834 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:45.479480 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4c88-1164-4ff2-b93f-f6e7ba8c4834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:45.479785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4c88-1164-4ff2-b93f-f6e7ba8c4834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:45.489723 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c4c88-1164-4ff2-b93f-f6e7ba8c4834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:45.491155 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c4c88-1164-4ff2-b93f-f6e7ba8c4834 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:45.502923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1334/2658] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:45.784317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:45.785850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:45.798441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d62bdc-e564-4962-9cb3-00f311107d40 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:45.804135 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:45.805257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:45.805741 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:45.806030 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] PATH_INFO: `/v3/projects/946d081e3c9143659e7f805a368fd6eb` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:45.807174 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=946d081e3c9143659e7f805a368fd6eb)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:45.809140 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:45.860675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946d081e3c9143659e7f805a368fd6eb', 'request_id': 'req-83192f60-c96e-42ce-a617-650ca1e4d135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.866253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946d081e3c9143659e7f805a368fd6eb', 'request_id': 'req-83192f60-c96e-42ce-a617-650ca1e4d135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.872507 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946d081e3c9143659e7f805a368fd6eb', 'request_id': 'req-83192f60-c96e-42ce-a617-650ca1e4d135'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.885427 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] The token cache is being invalidate because project 946d081e3c9143659e7f805a368fd6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:14:45.885719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.886334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-83192f60-c96e-42ce-a617-650ca1e4d135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:14:45.887792 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1325/2659] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:14:45 2022] DELETE /identity/v3/projects/946d081e3c9143659e7f805a368fd6eb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:45.892763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463895d-d4f4-4251-8bda-c55032fa0721 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:45.893269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463895d-d4f4-4251-8bda-c55032fa0721 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:45.893479 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463895d-d4f4-4251-8bda-c55032fa0721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:45.913843 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7463895d-d4f4-4251-8bda-c55032fa0721 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:45.915046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7463895d-d4f4-4251-8bda-c55032fa0721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:45.973496 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1335/2660] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:14:45.982770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8059d1-9e66-4025-b1b0-fd1baafe1018 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:45.983061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8059d1-9e66-4025-b1b0-fd1baafe1018 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:45.983338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8059d1-9e66-4025-b1b0-fd1baafe1018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:45.991899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:46.004667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0f8059d1-9e66-4025-b1b0-fd1baafe1018 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:46.005878 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0f8059d1-9e66-4025-b1b0-fd1baafe1018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:46.013900 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1326/2661] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:14:46.018144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463895d-d4f4-4251-8bda-c55032fa0721 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.028787 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463895d-d4f4-4251-8bda-c55032fa0721 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.033384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.033950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.034229 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.034504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.035799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:14:46.036147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.037738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.046180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:46.050053 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1336/2662] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:14:46.055069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8059d1-9e66-4025-b1b0-fd1baafe1018 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.064475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8059d1-9e66-4025-b1b0-fd1baafe1018 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.069413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.069974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.070280 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.070551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.071183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.073083 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.100380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1327/2663] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:14:46 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:14:46.104672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.114682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93afb1-94ef-42a0-ae81-2645db9f9beb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.170889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.171482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.171759 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.172058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.172671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.174617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.196578 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1337/2664] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:14:46 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:14:46.200786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.211860 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3dcc7-f7ea-4af2-b8ff-476f89cd79e0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.216429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.217016 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.217296 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.217568 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.218226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.219823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.230422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1328/2665] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:14:46.234394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.244545 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00df2a3-f422-4482-9174-96c8c932e568 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.249930 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3eb37-7def-4362-87da-22005a1ca017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.250524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3eb37-7def-4362-87da-22005a1ca017 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.250801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3eb37-7def-4362-87da-22005a1ca017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.251071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3eb37-7def-4362-87da-22005a1ca017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.251707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3eb37-7def-4362-87da-22005a1ca017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.253300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3eb37-7def-4362-87da-22005a1ca017 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.263979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1338/2666] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:14:46.268554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.279429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06021f-4576-40f4-9d33-910c1b4a76c9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.283982 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.284562 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.285411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.285685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] PATH_INFO: `/v3/projects/290447e117b841d19a9f552d2a99b281/users/7330527d2f2e4fa0b49dd1761866c05b/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.300109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=290447e117b841d19a9f552d2a99b281, user_id=7330527d2f2e4fa0b49dd1761866c05b, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.303419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.316214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1329/2667] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:14:46 2022] PUT /identity/v3/projects/290447e117b841d19a9f552d2a99b281/users/7330527d2f2e4fa0b49dd1761866c05b/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:46.326749 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b6402-c6a8-4ba7-a298-ac777efb0de6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.327096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b6402-c6a8-4ba7-a298-ac777efb0de6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.327371 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b6402-c6a8-4ba7-a298-ac777efb0de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.355202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cc7b6402-c6a8-4ba7-a298-ac777efb0de6 None None] MFA Rules not processed for user `7330527d2f2e4fa0b49dd1761866c05b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:46.356410 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc7b6402-c6a8-4ba7-a298-ac777efb0de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:46.419804 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1339/2668] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:14:46.424500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.484412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12274ba6-0345-4640-9ce3-8e7e3ba484fa admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.488928 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.489506 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.489783 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.490050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.490670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.492582 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.518152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1330/2669] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:14:46 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:14:46.523344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b6402-c6a8-4ba7-a298-ac777efb0de6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.534140 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b6402-c6a8-4ba7-a298-ac777efb0de6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.590185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.590781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.591057 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.591348 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.591994 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.593874 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.611289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:46.619252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1340/2670] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:14:46 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:14:46.623289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.633514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d4242-c933-410b-accc-690a2803dfe7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.638002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.638584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.638859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.639125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.639754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.641420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.651866 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1331/2671] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:14:46.655978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.666548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7849dd-2301-44f5-9d2d-f9d8d3262a24 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.671042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.671676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.671989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.672262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] PATH_INFO: `/v3/projects/63d3ee6b3fa644058f079d829080ca73/users/e98616bd37dd4b8ab6e129e8d8ab075e/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.687598 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d3ee6b3fa644058f079d829080ca73, user_id=e98616bd37dd4b8ab6e129e8d8ab075e, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.690305 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.703414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1341/2672] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:14:46 2022] PUT /identity/v3/projects/63d3ee6b3fa644058f079d829080ca73/users/e98616bd37dd4b8ab6e129e8d8ab075e/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:46.708421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.718184 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26b68-e1cc-41eb-b2d4-f657beb0cbf8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.774370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5efd2d-4516-4854-b2e1-8803d08305b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.775047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5efd2d-4516-4854-b2e1-8803d08305b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.775329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5efd2d-4516-4854-b2e1-8803d08305b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.775604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5efd2d-4516-4854-b2e1-8803d08305b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.776271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5efd2d-4516-4854-b2e1-8803d08305b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.777873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5efd2d-4516-4854-b2e1-8803d08305b0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.787662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1332/2673] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:14:46.791969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.803054 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b2405-4ab5-4d25-bab2-eb9c1275b314 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:46.807816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:46.808484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.808763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.809036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] PATH_INFO: `/v3/projects/63d3ee6b3fa644058f079d829080ca73/users/e98616bd37dd4b8ab6e129e8d8ab075e/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.810948 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d3ee6b3fa644058f079d829080ca73, user_id=e98616bd37dd4b8ab6e129e8d8ab075e, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:46.813599 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:46.824594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1342/2674] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:14:46 2022] PUT /identity/v3/projects/63d3ee6b3fa644058f079d829080ca73/users/e98616bd37dd4b8ab6e129e8d8ab075e/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:14:46.835037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f3b15-876b-4a17-8508-6edc48ffaba5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:46.835361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f3b15-876b-4a17-8508-6edc48ffaba5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:46.835644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f3b15-876b-4a17-8508-6edc48ffaba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:46.855808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:46.864811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d52f3b15-876b-4a17-8508-6edc48ffaba5 None None] MFA Rules not processed for user `e98616bd37dd4b8ab6e129e8d8ab075e`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:14:46.866669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d52f3b15-876b-4a17-8508-6edc48ffaba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:46.933765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1333/2675] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:14:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:14:46.945741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:46.947537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:14:47.005422 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e2203-51c1-4d9c-bf40-2d077aff76fb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:47.009099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:47.009698 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:47.009977 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:47.010250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:47.019773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:47.021242 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:47.030794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1343/2676] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:47.108907 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f3b15-876b-4a17-8508-6edc48ffaba5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:14:47.119514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f3b15-876b-4a17-8508-6edc48ffaba5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:14:47.123925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:14:47.124475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:14:47.124751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:14:47.125023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:14:47.182266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:14:47.183730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:14:47.193138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1334/2677] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:14:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:14:57.026708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:14:57.188537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=57 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:12.569542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:12.570799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:12.581700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d6cf2-e456-49f6-9587-7070708b88bf service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:12.585443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:12.586024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:12.586301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:12.586568 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:12.588273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:12.669221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:12.670587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:12.690280 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1344/2678] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:19.155195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:19.156845 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:19.281899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa50dda-efb9-4d62-93c4-32ab04663584 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:19.296618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:19.297248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:19.298057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:19.298329 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:19.300256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:19.315550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:19.317420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:19.330007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1335/2679] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:22.687158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=41 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:23.322198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:23.323600 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:23.379541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97e857-4a68-4e0c-abf6-7c4ff6aad54f service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:23.384478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:23.385120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:23.385402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:23.390168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:23.391646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:23.462785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:23.464263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:23.474020 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1345/2680] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:26.538840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:26.540319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:26.600367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:26.602502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:26.620111 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114e197-9f12-4dfa-9fb2-173b2b2237f1 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:26.624135 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0c96-321a-4f5f-864f-81878ab35c9c service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:26.625450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:26.626085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:26.626364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:26.626727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] PATH_INFO: `/v3/users/56835f0c8dc0416d9fa70662e40c856d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:26.627756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56835f0c8dc0416d9fa70662e40c856d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:26.629081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:26.629677 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:26.629882 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:26.629998 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:26.630212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:26.632974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:26.650147 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:26.652527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:26.665374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1336/2681] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:26.683967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56835f0c8dc0416d9fa70662e40c856d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:26.698664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56835f0c8dc0416d9fa70662e40c856d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:26.708710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56835f0c8dc0416d9fa70662e40c856d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:26.709758 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56835f0c8dc0416d9fa70662e40c856d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:26.720334 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2030787f-eaf6-4c5f-b915-0f5a77989dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56835f0c8dc0416d9fa70662e40c856d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:26.727621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1346/2682] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:26 2022] DELETE /identity/v3/users/56835f0c8dc0416d9fa70662e40c856d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:15:26.904265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:26.917696 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6c2b0-ff4d-44d2-9e61-18e938b26f29 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:26.974862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:26.976045 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:26.976333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:26.976605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] PATH_INFO: `/v3/projects/5b886dc365684ab1811d100eb5774619` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:26.977709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b886dc365684ab1811d100eb5774619)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:26.979628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:26.997102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:27.026623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b886dc365684ab1811d100eb5774619', 'request_id': 'req-1a723477-6b51-46da-93aa-e7ee96b0fcfc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:27.031937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b886dc365684ab1811d100eb5774619', 'request_id': 'req-1a723477-6b51-46da-93aa-e7ee96b0fcfc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:27.038821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b886dc365684ab1811d100eb5774619', 'request_id': 'req-1a723477-6b51-46da-93aa-e7ee96b0fcfc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:27.052133 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] The token cache is being invalidate because project 5b886dc365684ab1811d100eb5774619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:27.052579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:27.053376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-1a723477-6b51-46da-93aa-e7ee96b0fcfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:27.055659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1337/2683] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:26 2022] DELETE /identity/v3/projects/5b886dc365684ab1811d100eb5774619 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:15:27.060729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf986d-5904-4fd9-8da2-be42357da469 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.061024 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf986d-5904-4fd9-8da2-be42357da469 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.061307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf986d-5904-4fd9-8da2-be42357da469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.082104 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6ecf986d-5904-4fd9-8da2-be42357da469 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:27.083304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6ecf986d-5904-4fd9-8da2-be42357da469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:27.102590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:27.152734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1347/2684] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:27.160823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7393b-fe55-4a30-9293-0b6be1e686f2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.161114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7393b-fe55-4a30-9293-0b6be1e686f2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.161394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7393b-fe55-4a30-9293-0b6be1e686f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.182352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a2c7393b-fe55-4a30-9293-0b6be1e686f2 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:27.183557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a2c7393b-fe55-4a30-9293-0b6be1e686f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:27.191544 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1338/2685] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:27.195867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf986d-5904-4fd9-8da2-be42357da469 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:27.205435 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf986d-5904-4fd9-8da2-be42357da469 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:27.211195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:27.211790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.212102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.212401 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.213041 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:15:27.213341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:27.214906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:27.227047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1348/2686] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:15:27.231285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7393b-fe55-4a30-9293-0b6be1e686f2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:27.240669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7393b-fe55-4a30-9293-0b6be1e686f2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:27.245481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:27.246066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.246372 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.246645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.247278 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:27.249216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:27.276646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1339/2687] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:27 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:27.282027 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:27.292832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1816e9-4766-4cb1-9cf4-51fc1c84de8b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:27.349248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:27.349877 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.350154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.350525 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.351827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:27.353837 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:27.374072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1349/2688] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:27 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:27.378252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:27.388605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee967-e285-44ac-80ad-91979c38bcc0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:27.393332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:27.393925 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.394207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.394483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.395123 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:27.396764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:27.408051 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1340/2689] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:27.412223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:27.422742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b4235-9be1-49e1-be46-e7009be120f4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:27.427419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5298cd-9e10-47fc-8e7c-938f5fd36de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:27.428044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5298cd-9e10-47fc-8e7c-938f5fd36de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.428322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5298cd-9e10-47fc-8e7c-938f5fd36de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.428602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5298cd-9e10-47fc-8e7c-938f5fd36de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.429235 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5298cd-9e10-47fc-8e7c-938f5fd36de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:27.430813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5298cd-9e10-47fc-8e7c-938f5fd36de5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:27.441485 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1350/2690] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:27.445995 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:27.455945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaa9f5-6040-4666-95bb-2eb0e1df5d48 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:27.461433 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:27.462063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.462345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.462623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] PATH_INFO: `/v3/projects/5d69675576dc4c7fa5cede0f99db8e32/users/4ad5a3ee5ab545e2893d4f6365f8f939/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.477944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d69675576dc4c7fa5cede0f99db8e32, user_id=4ad5a3ee5ab545e2893d4f6365f8f939, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:27.480636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:27.494139 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1341/2691] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:27 2022] PUT /identity/v3/projects/5d69675576dc4c7fa5cede0f99db8e32/users/4ad5a3ee5ab545e2893d4f6365f8f939/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:27.499145 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.499430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.499708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.518102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:27.519283 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:27.578044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1351/2692] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:27.588346 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:27.590050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:27.646423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c93465-e7a5-48a8-9884-356d8073261d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:27.651035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:27.651625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:27.651934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:27.652212 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:27.661069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:27.662518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:27.672023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1342/2693] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:29.497608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:29.499194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:29.523831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f583b76-167b-4647-92c7-9ec5ecc876ba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:29.529889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a800e-c9e3-433f-bb06-4c13bde46212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:29.530515 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a800e-c9e3-433f-bb06-4c13bde46212 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:29.530795 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a800e-c9e3-433f-bb06-4c13bde46212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:29.531069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a800e-c9e3-433f-bb06-4c13bde46212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:29.536387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-315a800e-c9e3-433f-bb06-4c13bde46212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:29.562022 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a800e-c9e3-433f-bb06-4c13bde46212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:29.565713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a800e-c9e3-433f-bb06-4c13bde46212 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:29.583407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1352/2694] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:31.182483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:31.184144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:31.214948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbd621-e57c-48cb-8175-c014a7fb97c0 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:31.222940 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:31.223238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:31.223522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:31.224148 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] PATH_INFO: `/v3/users/7330527d2f2e4fa0b49dd1761866c05b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:31.226308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7330527d2f2e4fa0b49dd1761866c05b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:31.229388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:31.312577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:31.321847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7330527d2f2e4fa0b49dd1761866c05b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:31.341314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7330527d2f2e4fa0b49dd1761866c05b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:31.342408 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7330527d2f2e4fa0b49dd1761866c05b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:31.353971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7330527d2f2e4fa0b49dd1761866c05b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:31.354754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7330527d2f2e4fa0b49dd1761866c05b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:31.397017 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1343/2695] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:31 2022] DELETE /identity/v3/users/7330527d2f2e4fa0b49dd1761866c05b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:31.402213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:31.402686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:31.402786 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:31.443634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:31.445454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:31.465197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:31.571352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1353/2696] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:31.595438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:31.597386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:31.721810 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cb653-9325-4019-9b0b-ddfd4bc4b8ec admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:31.729859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:31.730391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:31.730926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:31.731213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:31.741513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:31.743217 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:31.770958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1344/2697] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:32.298722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:32.300546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:32.325264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc359c22-a440-4735-8a57-6d5cf8566f51 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:32.338212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:32.340682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:32.342513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:32.343058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] PATH_INFO: `/v3/projects/290447e117b841d19a9f552d2a99b281` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:32.345687 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=290447e117b841d19a9f552d2a99b281)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:32.354079 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:32.454263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '290447e117b841d19a9f552d2a99b281', 'request_id': 'req-baff47b7-f697-4a79-9e2b-316fdb897758'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.484165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '290447e117b841d19a9f552d2a99b281', 'request_id': 'req-baff47b7-f697-4a79-9e2b-316fdb897758'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.508342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '290447e117b841d19a9f552d2a99b281', 'request_id': 'req-baff47b7-f697-4a79-9e2b-316fdb897758'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.538776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] The token cache is being invalidate because project 290447e117b841d19a9f552d2a99b281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:32.539081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.539768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.547575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1354/2698] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:32 2022] DELETE /identity/v3/projects/290447e117b841d19a9f552d2a99b281 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:32.548902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:32.566855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa688f2f-e9e8-4fb3-9320-901f69bafb25 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:32.668763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:32.670682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:32.670967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:32.671359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] PATH_INFO: `/v3/users/e98616bd37dd4b8ab6e129e8d8ab075e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:32.673580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98616bd37dd4b8ab6e129e8d8ab075e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:32.679333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:32.793760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98616bd37dd4b8ab6e129e8d8ab075e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.812701 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:32.818952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98616bd37dd4b8ab6e129e8d8ab075e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.823345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98616bd37dd4b8ab6e129e8d8ab075e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.841381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98616bd37dd4b8ab6e129e8d8ab075e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.842044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d2b3d92-5ea7-46a0-8723-1584684e3439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98616bd37dd4b8ab6e129e8d8ab075e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:32.869981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1345/2699] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:32 2022] DELETE /identity/v3/users/e98616bd37dd4b8ab6e129e8d8ab075e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:33.367256 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:33.368952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:33.393519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff47b7-f697-4a79-9e2b-316fdb897758 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:33.480411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:33.481182 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:33.481593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:33.481932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] PATH_INFO: `/v3/projects/63d3ee6b3fa644058f079d829080ca73` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:33.483197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d3ee6b3fa644058f079d829080ca73)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:33.485408 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:33.543816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:33.558372 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d3ee6b3fa644058f079d829080ca73', 'request_id': 'req-fc857be6-f445-4924-8ff3-85aae862cac0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:33.565668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d3ee6b3fa644058f079d829080ca73', 'request_id': 'req-fc857be6-f445-4924-8ff3-85aae862cac0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:33.577128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d3ee6b3fa644058f079d829080ca73', 'request_id': 'req-fc857be6-f445-4924-8ff3-85aae862cac0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:33.610519 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] The token cache is being invalidate because project 63d3ee6b3fa644058f079d829080ca73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:33.612353 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:33.614502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc857be6-f445-4924-8ff3-85aae862cac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:33.617827 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1355/2700] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:33 2022] DELETE /identity/v3/projects/63d3ee6b3fa644058f079d829080ca73 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:33.622779 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fbbaa-d3a3-4d83-aea6-b236100563dd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:33.623255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fbbaa-d3a3-4d83-aea6-b236100563dd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:33.623520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fbbaa-d3a3-4d83-aea6-b236100563dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:33.660732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fb4fbbaa-d3a3-4d83-aea6-b236100563dd None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:33.664438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fb4fbbaa-d3a3-4d83-aea6-b236100563dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:33.766542 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1346/2701] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:33.768272 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956aa34-3698-4342-a410-82678580729d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:33.768929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956aa34-3698-4342-a410-82678580729d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:33.768929 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956aa34-3698-4342-a410-82678580729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:33.797906 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a956aa34-3698-4342-a410-82678580729d None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:33.799486 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a956aa34-3698-4342-a410-82678580729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:33.816143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1356/2702] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:33.816776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fbbaa-d3a3-4d83-aea6-b236100563dd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:33.827098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fbbaa-d3a3-4d83-aea6-b236100563dd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:33.836680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:33.837310 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:33.837591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:33.837867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:33.838527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:15:33.838884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:33.842685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:33.865300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:33.868337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:33.868769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:33.877828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1347/2703] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:15:33.882780 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:33.895297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6830108-b43c-4cd0-a9ee-7b3d8ca6b733 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:33.907527 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:33.908181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:33.908469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:33.909063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:33.909714 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:33.911637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:33.939979 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] MFA Rules not processed for user `4ad5a3ee5ab545e2893d4f6365f8f939`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:33.947343 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:33.986884 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1348/2704] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:33 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:33.987730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.076540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1357/2705] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:34.090825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.092265 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:34.093790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4127a3-44b9-4631-b20b-423e1e974085 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.098606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.099220 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.099504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.099781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.100491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.102437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.129591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1349/2706] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:33 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:34.134185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.146050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67819a-73ef-4c32-9859-247ace331e77 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.152994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.153624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.154257 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.154909 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.155426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.157783 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.181439 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1350/2707] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:15:34.182392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.195238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c8309-4cc1-4ca4-9197-c0708b698ba2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.196682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927a31d-4070-426f-896d-641faacca65e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.203557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.205128 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.205412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.205771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.206330 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.208647 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.229305 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1351/2708] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:34.235219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.248960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871d9a7-1c28-46cc-9967-578111bbc659 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.253984 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820623c8-f182-4c3e-b32a-9188ae43506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.255273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820623c8-f182-4c3e-b32a-9188ae43506d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.255273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820623c8-f182-4c3e-b32a-9188ae43506d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.255619 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820623c8-f182-4c3e-b32a-9188ae43506d admin admin] PATH_INFO: `/v3/projects/68b8db5909b746749b713b47fcd619be/users/2f734372033e4813816c05b508aedb04/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.262221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:34.269189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:34.280886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820623c8-f182-4c3e-b32a-9188ae43506d admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b8db5909b746749b713b47fcd619be, user_id=2f734372033e4813816c05b508aedb04, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.283695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820623c8-f182-4c3e-b32a-9188ae43506d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.287363 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.288660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.288951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.289276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.300052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1352/2709] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:34 2022] PUT /identity/v3/projects/68b8db5909b746749b713b47fcd619be/users/2f734372033e4813816c05b508aedb04/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:34.312755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a211-33ca-45fb-8ec2-72f5962a8b89 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.313037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a211-33ca-45fb-8ec2-72f5962a8b89 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.313309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a211-33ca-45fb-8ec2-72f5962a8b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.342574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0624a211-33ca-45fb-8ec2-72f5962a8b89 None None] MFA Rules not processed for user `2f734372033e4813816c05b508aedb04`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:34.343823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0624a211-33ca-45fb-8ec2-72f5962a8b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:34.355487 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.357129 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.366185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1358/2710] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:15:34.413271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1353/2711] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:15:34.417889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.429099 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ec8cf-64db-47c1-8454-b8efc94e0255 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.492938 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.494295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.494684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.494824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.495470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.497489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.527872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1359/2712] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:34 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:34.532098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624a211-33ca-45fb-8ec2-72f5962a8b89 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.544204 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624a211-33ca-45fb-8ec2-72f5962a8b89 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.617645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.618387 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.618809 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.619109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.620074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.622331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.649756 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1354/2713] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:34 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:34.653584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.664695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6f8e-8abf-4ae7-a23f-0434e806322f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.670792 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.671596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.671955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.672264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.672943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.674552 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.687646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1360/2714] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:15:34.691224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.702019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd890f-fc30-44d1-97e0-618f9f9cc1cb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.707869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.708872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.709244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.709528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] PATH_INFO: `/v3/projects/e1f4a9334189480c866f282e74ba012e/users/25608d6523254d69994f06da17cb53f7/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.727670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f4a9334189480c866f282e74ba012e, user_id=25608d6523254d69994f06da17cb53f7, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.730754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.749352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1355/2715] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:34 2022] PUT /identity/v3/projects/e1f4a9334189480c866f282e74ba012e/users/25608d6523254d69994f06da17cb53f7/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:34.753421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.764611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4984552-2b18-47bb-b226-29d8199b06d3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.842084 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bbef7-919d-4e8e-b200-d7db5f83c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.842960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bbef7-919d-4e8e-b200-d7db5f83c204 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.843298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bbef7-919d-4e8e-b200-d7db5f83c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.843581 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bbef7-919d-4e8e-b200-d7db5f83c204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.844288 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bbef7-919d-4e8e-b200-d7db5f83c204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.845966 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bbef7-919d-4e8e-b200-d7db5f83c204 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.848951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:34.863452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:34.864560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1361/2716] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:34.874916 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dcd0d-3e93-4e9a-aa49-b337b45edcd3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:34.881445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:34.883324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.883665 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.884272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] PATH_INFO: `/v3/projects/e1f4a9334189480c866f282e74ba012e/users/25608d6523254d69994f06da17cb53f7/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.887059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f4a9334189480c866f282e74ba012e, user_id=25608d6523254d69994f06da17cb53f7, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:34.889755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:34.901251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1356/2717] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:34 2022] PUT /identity/v3/projects/e1f4a9334189480c866f282e74ba012e/users/25608d6523254d69994f06da17cb53f7/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:34.911611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:34.911980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:34.912248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:34.943115 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] MFA Rules not processed for user `25608d6523254d69994f06da17cb53f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:34.945040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:35.031432 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1362/2718] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:35.043565 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:35.045411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:35.101152 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:35.139912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fb1da-3798-4887-808b-11eef26a6687 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:35.143636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e8ecb-d6ab-4a97-9de8-633566256d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:35.144297 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e8ecb-d6ab-4a97-9de8-633566256d99 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:35.144580 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e8ecb-d6ab-4a97-9de8-633566256d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:35.144859 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e8ecb-d6ab-4a97-9de8-633566256d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:35.155614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e8ecb-d6ab-4a97-9de8-633566256d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:35.157122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e8ecb-d6ab-4a97-9de8-633566256d99 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:35.180114 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1357/2719] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:35.239460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:35.241708 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:35.256101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af65f-ccdb-42f7-bf34-2d2afb4b955d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:35.262797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:35.263466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:35.263742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:35.264093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:35.363849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:35.365778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:35.381529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1363/2720] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:35.855468 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:35.856052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:35.856052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:35.876616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] MFA Rules not processed for user `4ad5a3ee5ab545e2893d4f6365f8f939`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:35.877823 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:35.958523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1358/2721] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:36.589192 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:36.590580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:36.657588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b684775-b070-45e1-ad8e-c735a27bce74 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:36.661151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:36.661757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:36.662037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:36.662310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:36.664695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:36.675144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:36.677196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:36.687102 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1364/2722] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:37.055207 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:37.056803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:37.169867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5078a-d763-4db2-affd-79cd77f3df37 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:37.179218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741f823-67cc-4de0-ad1d-39bd98c4948e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:37.180023 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f823-67cc-4de0-ad1d-39bd98c4948e service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:37.180348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f823-67cc-4de0-ad1d-39bd98c4948e service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:37.180654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f823-67cc-4de0-ad1d-39bd98c4948e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:37.182703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b741f823-67cc-4de0-ad1d-39bd98c4948e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:37.196606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741f823-67cc-4de0-ad1d-39bd98c4948e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:37.198567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741f823-67cc-4de0-ad1d-39bd98c4948e service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:37.224524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1359/2723] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:38.991029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:39.035533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:39.078419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e02f7-6eb9-4ab8-9ccd-420f22d993bb service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:39.084168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:39.085668 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:39.085888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:39.086213 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] PATH_INFO: `/v3/users/2f734372033e4813816c05b508aedb04` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:39.087672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f734372033e4813816c05b508aedb04)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:39.089750 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:39.145331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f734372033e4813816c05b508aedb04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.159513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f734372033e4813816c05b508aedb04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.166718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f734372033e4813816c05b508aedb04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.167426 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f734372033e4813816c05b508aedb04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.174461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f734372033e4813816c05b508aedb04'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.181128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1365/2724] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:38 2022] DELETE /identity/v3/users/2f734372033e4813816c05b508aedb04 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:39.185767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:39.186318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:39.186318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:39.206020 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:39.207440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:39.274052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1360/2725] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:15:39.286067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:39.289736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:39.365222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd043fc-c714-497c-99cc-bf28c0decba6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:39.369761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:39.370380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:39.370664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:39.370939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:39.380430 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:39.382501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:39.393374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1366/2726] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:39.821370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:39.823654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:39.839420 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfacb-1907-4581-b4e3-994b21465da6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:39.849534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:39.850351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:39.850902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:39.851353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] PATH_INFO: `/v3/projects/68b8db5909b746749b713b47fcd619be` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:39.852616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b8db5909b746749b713b47fcd619be)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:39.854659 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:39.856577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:39.915522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b8db5909b746749b713b47fcd619be', 'request_id': 'req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.924903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b8db5909b746749b713b47fcd619be', 'request_id': 'req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.933467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b8db5909b746749b713b47fcd619be', 'request_id': 'req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.949572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] The token cache is being invalidate because project 68b8db5909b746749b713b47fcd619be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:39.949868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.950712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4a957c2e-bfeb-43e0-9233-2a2a59f70fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:39.952334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1361/2727] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:39 2022] DELETE /identity/v3/projects/68b8db5909b746749b713b47fcd619be => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:39.957704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:39.968785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e288d-37cd-4816-8a6f-133bf116aa51 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:40.029858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:40.051333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:40.052247 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:40.053069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:40.053950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] PATH_INFO: `/v3/users/25608d6523254d69994f06da17cb53f7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:40.055262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=25608d6523254d69994f06da17cb53f7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:40.057628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:40.124025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25608d6523254d69994f06da17cb53f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.143085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25608d6523254d69994f06da17cb53f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.155543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25608d6523254d69994f06da17cb53f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.156282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25608d6523254d69994f06da17cb53f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.167124 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25608d6523254d69994f06da17cb53f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.175440 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1367/2728] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:39 2022] DELETE /identity/v3/users/25608d6523254d69994f06da17cb53f7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:40.182981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:40.183472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:40.183712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:40.242960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] MFA Rules not processed for user `f0819d4fee38422cb796aab2df4069e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:40.249567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:40.355472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1362/2729] 10.222.0.52 () {58 vars in 967 bytes} [Thu Oct 6 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:40.441459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:40.443807 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:40.549191 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e946-5790-46ce-a760-46d5f20e96ca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:40.555653 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0f5df-fbd6-4196-8675-597b12803aec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5aa06f9803465bb102482d351adeac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:40.556307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0f5df-fbd6-4196-8675-597b12803aec service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:40.556592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0f5df-fbd6-4196-8675-597b12803aec service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:40.557047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0f5df-fbd6-4196-8675-597b12803aec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:40.566067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0f5df-fbd6-4196-8675-597b12803aec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:40.567654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0f5df-fbd6-4196-8675-597b12803aec service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:40.578734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1368/2730] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Oct 6 11:15:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 11:15:40.623636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:40.626135 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:40.720758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06764fa8-5600-4c6d-9f68-5cd624fa1b73 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:40.725988 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:40.726626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:40.726902 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:40.727172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] PATH_INFO: `/v3/projects/e1f4a9334189480c866f282e74ba012e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:40.728464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f4a9334189480c866f282e74ba012e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:40.730375 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:40.733025 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:40.785965 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f4a9334189480c866f282e74ba012e', 'request_id': 'req-aed5d54d-624a-4efb-b399-9d3210d3445d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.793151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f4a9334189480c866f282e74ba012e', 'request_id': 'req-aed5d54d-624a-4efb-b399-9d3210d3445d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.800406 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f4a9334189480c866f282e74ba012e', 'request_id': 'req-aed5d54d-624a-4efb-b399-9d3210d3445d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.816981 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] The token cache is being invalidate because project e1f4a9334189480c866f282e74ba012e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:40.817181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.817805 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-aed5d54d-624a-4efb-b399-9d3210d3445d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:40.820074 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1363/2731] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:40 2022] DELETE /identity/v3/projects/e1f4a9334189480c866f282e74ba012e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:40.825429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70084-bd83-43cd-ab12-e4f0c8e42e87 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:40.825712 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70084-bd83-43cd-ab12-e4f0c8e42e87 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:40.825983 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70084-bd83-43cd-ab12-e4f0c8e42e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:40.846228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bdf70084-bd83-43cd-ab12-e4f0c8e42e87 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:40.847419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bdf70084-bd83-43cd-ab12-e4f0c8e42e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:40.884677 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:40.921282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1369/2732] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:15:40.929541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa8599-ee7f-4c96-b7a1-0c28f23a96c1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:40.929969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa8599-ee7f-4c96-b7a1-0c28f23a96c1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:40.930073 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa8599-ee7f-4c96-b7a1-0c28f23a96c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:40.951961 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b7aa8599-ee7f-4c96-b7a1-0c28f23a96c1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:40.953168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b7aa8599-ee7f-4c96-b7a1-0c28f23a96c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:40.961283 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1364/2733] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:40.965411 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70084-bd83-43cd-ab12-e4f0c8e42e87 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:40.975958 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70084-bd83-43cd-ab12-e4f0c8e42e87 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:40.980589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:40.981544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:40.981544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:40.982260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:40.982873 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:15:40.982984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:40.985172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:40.997880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1370/2734] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:15:41.002183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa8599-ee7f-4c96-b7a1-0c28f23a96c1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.012221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa8599-ee7f-4c96-b7a1-0c28f23a96c1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.016958 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.017634 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.017978 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.018313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.019012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.021004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.051081 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1365/2735] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:41 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:41.056048 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.066151 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9f489-1bed-4cc9-a230-e009f54f2357 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.128604 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.129439 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.129779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.130097 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.130989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.132969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.157367 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1371/2736] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:41 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:41.161560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.172718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2589a-e318-4283-ba5a-8e60762dd933 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.177378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.177952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.178229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.179060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.179919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.181511 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.191658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1366/2737] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:15:41.196036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.206246 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1240fb-a2bd-4abc-84af-fd3476ec24ea admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.211052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390f63a-3e3f-435a-9f44-5dd5c7598038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.211667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390f63a-3e3f-435a-9f44-5dd5c7598038 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.211989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390f63a-3e3f-435a-9f44-5dd5c7598038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.212269 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390f63a-3e3f-435a-9f44-5dd5c7598038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.212916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390f63a-3e3f-435a-9f44-5dd5c7598038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.214620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390f63a-3e3f-435a-9f44-5dd5c7598038 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.226181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1372/2738] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:41.230646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.242651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e805705-9a7d-43ba-a0b9-2b1dac0340a7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.247265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.247957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.248238 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.248513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] PATH_INFO: `/v3/projects/59790841739e486ebfa39b465f009182/users/0e66c4020a3649f19fc05f89e343d4d9/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.264755 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59790841739e486ebfa39b465f009182, user_id=0e66c4020a3649f19fc05f89e343d4d9, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.267376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.280944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1367/2739] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:41 2022] PUT /identity/v3/projects/59790841739e486ebfa39b465f009182/users/0e66c4020a3649f19fc05f89e343d4d9/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:41.292741 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec7902-a5ba-401a-860e-5726078096a2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.293029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec7902-a5ba-401a-860e-5726078096a2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.293303 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec7902-a5ba-401a-860e-5726078096a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.323000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f6ec7902-a5ba-401a-860e-5726078096a2 None None] MFA Rules not processed for user `0e66c4020a3649f19fc05f89e343d4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:41.324394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f6ec7902-a5ba-401a-860e-5726078096a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:41.393467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1373/2740] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:41.399028 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.462394 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a102c5-daad-4fff-9d82-ec740c4a3a1d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.467004 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.467601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.467903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.468177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.468808 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.470694 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.498627 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1368/2741] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:41 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:41.502751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec7902-a5ba-401a-860e-5726078096a2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.513233 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec7902-a5ba-401a-860e-5726078096a2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.544575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:41.574665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.575195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.575477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.575752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.576416 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.578340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.599384 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1374/2742] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:41 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:41.604065 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.614380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c982e-aec4-4508-a11a-f60c3da2c5fc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.619664 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.620294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.620606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.620877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.621522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.623143 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.636463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1369/2743] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:41.640694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.652282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b69030-655e-4fdd-bda9-1aa332819c46 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.656931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.657584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.657863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.658139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] PATH_INFO: `/v3/projects/011475c7b5c5448d94724544cb68e25b/users/12a7ca4ff37a4c54948902379e1304b6/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.674036 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=011475c7b5c5448d94724544cb68e25b, user_id=12a7ca4ff37a4c54948902379e1304b6, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.676729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.692632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1375/2744] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:41 2022] PUT /identity/v3/projects/011475c7b5c5448d94724544cb68e25b/users/12a7ca4ff37a4c54948902379e1304b6/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:41.697096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.702938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:41.709203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19952ef-0a94-4ffb-8fdb-70be2107e912 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.778543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af57c16-22d0-4747-a369-2293e957ee55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.779518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af57c16-22d0-4747-a369-2293e957ee55 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.779894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af57c16-22d0-4747-a369-2293e957ee55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.780241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af57c16-22d0-4747-a369-2293e957ee55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.781275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af57c16-22d0-4747-a369-2293e957ee55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.782639 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af57c16-22d0-4747-a369-2293e957ee55 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.797234 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1370/2745] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:41.801337 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.814105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef864864-79b0-481a-868c-1c63b63f2d9d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:41.819154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:41.819869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.820168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.820589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] PATH_INFO: `/v3/projects/011475c7b5c5448d94724544cb68e25b/users/12a7ca4ff37a4c54948902379e1304b6/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.822563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011475c7b5c5448d94724544cb68e25b, user_id=12a7ca4ff37a4c54948902379e1304b6, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:41.825582 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:41.837756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1376/2746] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:41 2022] PUT /identity/v3/projects/011475c7b5c5448d94724544cb68e25b/users/12a7ca4ff37a4c54948902379e1304b6/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:41.847652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:41.847972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:41.848251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:41.888692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] MFA Rules not processed for user `12a7ca4ff37a4c54948902379e1304b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:41.890063 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:41.974557 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1371/2747] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:41 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:41.982074 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:41.984128 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:42.050533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef42fe5-7a7c-4131-affc-f7f9a7e7b861 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:42.056524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:42.057157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:42.057419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:42.057693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:42.073881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:42.075376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:42.089841 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1377/2748] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:41 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:42.140436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:42.141883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:42.154829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597071d-703c-45f4-8b42-0eb60974adf7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:42.158513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccec1cb-76b4-4f61-a2f2-cb5872d8c93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:42.159120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccec1cb-76b4-4f61-a2f2-cb5872d8c93c service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:42.159402 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccec1cb-76b4-4f61-a2f2-cb5872d8c93c service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:42.159681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccec1cb-76b4-4f61-a2f2-cb5872d8c93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:42.221868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccec1cb-76b4-4f61-a2f2-cb5872d8c93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:42.223601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccec1cb-76b4-4f61-a2f2-cb5872d8c93c service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:42.234078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1372/2749] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:45.810364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:45.875198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6627d4a-62da-4adc-a181-ac127bb0a5b6 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:45.879692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:45.880356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:45.880635 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:45.880907 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] PATH_INFO: `/v3/users/0e66c4020a3649f19fc05f89e343d4d9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:45.882769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e66c4020a3649f19fc05f89e343d4d9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:45.885249 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:45.920336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:45.948126 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e66c4020a3649f19fc05f89e343d4d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:45.968210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e66c4020a3649f19fc05f89e343d4d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:45.985575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e66c4020a3649f19fc05f89e343d4d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:45.985575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e66c4020a3649f19fc05f89e343d4d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:45.993558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e66c4020a3649f19fc05f89e343d4d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.000477 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1378/2750] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:45 2022] DELETE /identity/v3/users/0e66c4020a3649f19fc05f89e343d4d9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:46.004862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:46.005781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:46.006060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:46.031339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:46.033240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:46.052536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:46.101488 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1373/2751] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:46.110952 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:46.112782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:46.181564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46993b4a-b117-479c-bd26-0552558959ea admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:46.185222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:46.185825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:46.186098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:46.186368 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:46.196610 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:46.198038 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:46.207400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1379/2752] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:46.563465 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:46.566087 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:46.592440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73718e7a-d8de-43b2-b677-2775c64d3b28 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:46.614436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:46.615292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:46.615496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:46.616553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] PATH_INFO: `/v3/projects/59790841739e486ebfa39b465f009182` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:46.618192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59790841739e486ebfa39b465f009182)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:46.620261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:46.712043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59790841739e486ebfa39b465f009182', 'request_id': 'req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.724381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59790841739e486ebfa39b465f009182', 'request_id': 'req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.735432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59790841739e486ebfa39b465f009182', 'request_id': 'req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.750699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] The token cache is being invalidate because project 59790841739e486ebfa39b465f009182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:46.750996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.751620 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.753327 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1374/2753] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:46 2022] DELETE /identity/v3/projects/59790841739e486ebfa39b465f009182 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:46.758093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:46.773586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8bd43-8e7e-424b-9aac-c3b311606145 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:46.839266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:46.841181 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:46.841646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:46.842357 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] PATH_INFO: `/v3/users/12a7ca4ff37a4c54948902379e1304b6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:46.844077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a7ca4ff37a4c54948902379e1304b6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:46.846202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:46.930424 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a7ca4ff37a4c54948902379e1304b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.932735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:46.948218 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a7ca4ff37a4c54948902379e1304b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.957003 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a7ca4ff37a4c54948902379e1304b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.957673 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a7ca4ff37a4c54948902379e1304b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.966550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-00a7a223-3420-4499-b993-f7106b68d9df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a7ca4ff37a4c54948902379e1304b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:46.973609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1380/2754] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:46 2022] DELETE /identity/v3/users/12a7ca4ff37a4c54948902379e1304b6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:47.270625 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:47.273175 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:47.301968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023ccd7-216c-4df6-a17a-a9571a6bf9a5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:47.374986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:47.375616 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.375994 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.376294 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] PATH_INFO: `/v3/projects/011475c7b5c5448d94724544cb68e25b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.377396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011475c7b5c5448d94724544cb68e25b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:47.379309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:47.429480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:47.431158 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011475c7b5c5448d94724544cb68e25b', 'request_id': 'req-ab475817-7e40-40fa-9050-32191f7e3fe6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:47.438523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011475c7b5c5448d94724544cb68e25b', 'request_id': 'req-ab475817-7e40-40fa-9050-32191f7e3fe6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:47.446350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011475c7b5c5448d94724544cb68e25b', 'request_id': 'req-ab475817-7e40-40fa-9050-32191f7e3fe6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:47.460248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] The token cache is being invalidate because project 011475c7b5c5448d94724544cb68e25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:47.460668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:47.461652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ab475817-7e40-40fa-9050-32191f7e3fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:47.464223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1375/2755] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:47 2022] DELETE /identity/v3/projects/011475c7b5c5448d94724544cb68e25b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:47.469123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceb054-0dde-48d3-88f1-b4768f8c20be None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.469443 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceb054-0dde-48d3-88f1-b4768f8c20be None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.469744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceb054-0dde-48d3-88f1-b4768f8c20be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.490481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e6ceb054-0dde-48d3-88f1-b4768f8c20be None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:47.492241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e6ceb054-0dde-48d3-88f1-b4768f8c20be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:47.552070 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1381/2756] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:47.561903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90a4c-954e-450b-a521-1e036c2d7a36 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.562197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90a4c-954e-450b-a521-1e036c2d7a36 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.562476 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90a4c-954e-450b-a521-1e036c2d7a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.584727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-67a90a4c-954e-450b-a521-1e036c2d7a36 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:47.586381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-67a90a4c-954e-450b-a521-1e036c2d7a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:47.594758 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1376/2757] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:47.599017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceb054-0dde-48d3-88f1-b4768f8c20be None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:47.609858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceb054-0dde-48d3-88f1-b4768f8c20be None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:47.615051 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:47.615655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.616628 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.616904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.617563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:15:47.617870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:47.619458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:47.633492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1382/2758] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:15:47.638084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a90a4c-954e-450b-a521-1e036c2d7a36 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:47.649270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a90a4c-954e-450b-a521-1e036c2d7a36 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:47.654196 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:47.654793 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.655075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.655349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.656014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:47.658097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:47.689349 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1377/2759] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:47 2022] POST /identity/v3/projects => generated 359 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:47.693580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:47.706456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681e50d-ba8d-4407-b240-79b33e964307 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:47.773943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:47.774728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.775077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.775419 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.776195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:47.778252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:47.809109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1383/2760] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:47 2022] POST /identity/v3/users => generated 280 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:47.810326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:47.823097 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ce94e-decd-4784-a1b6-22d20617b512 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:47.828704 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:47.829491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.829776 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.830060 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.830693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:47.832353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:47.859534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1378/2761] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:47.860470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:47.871920 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addfc25-42d1-46b0-9445-ec04d93a2bbf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:47.876959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60be9e-9b5b-4c5e-8090-53411b82930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:47.877701 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60be9e-9b5b-4c5e-8090-53411b82930a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.878178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60be9e-9b5b-4c5e-8090-53411b82930a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.880406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60be9e-9b5b-4c5e-8090-53411b82930a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.880406 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60be9e-9b5b-4c5e-8090-53411b82930a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:47.882254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60be9e-9b5b-4c5e-8090-53411b82930a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:47.899331 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1384/2762] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:15:47.901718 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:47.916523 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9b29f-c445-4e41-b26d-1d1c5c62fa27 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:47.924567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:47.925360 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:47.925826 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:47.926256 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] PATH_INFO: `/v3/projects/a08df817802f4957ae88ee23847c3535/users/5e27d68ff9244f00be6bde691671affd/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:47.957456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08df817802f4957ae88ee23847c3535, user_id=5e27d68ff9244f00be6bde691671affd, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:47.961185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:47.999413 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1379/2763] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:47 2022] PUT /identity/v3/projects/a08df817802f4957ae88ee23847c3535/users/5e27d68ff9244f00be6bde691671affd/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:48.010926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee34d57-aaba-4ee1-bb4a-519fb7047c2f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:48.011244 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee34d57-aaba-4ee1-bb4a-519fb7047c2f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:48.011526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee34d57-aaba-4ee1-bb4a-519fb7047c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:48.061119 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ee34d57-aaba-4ee1-bb4a-519fb7047c2f None None] MFA Rules not processed for user `5e27d68ff9244f00be6bde691671affd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:48.062499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ee34d57-aaba-4ee1-bb4a-519fb7047c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:48.096845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:48.215735 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1385/2764] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:48.261279 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:48.263024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:48.338983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f564eb-c289-4fdb-8ef1-5be3aa80d23a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:48.343232 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:48.343868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:48.344554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:48.344828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:48.356189 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:48.357715 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:48.368546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1380/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:48.384592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee34d57-aaba-4ee1-bb4a-519fb7047c2f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:48.435763 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:48.437217 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:48.465421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee34d57-aaba-4ee1-bb4a-519fb7047c2f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:48.470139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:48.470781 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:48.471072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:48.471350 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] PATH_INFO: `/v3/users/5e27d68ff9244f00be6bde691671affd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:48.472832 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e27d68ff9244f00be6bde691671affd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:48.474770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:48.512481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9934ba-5ddc-47af-a7f6-6b6c769f6dce service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:48.517292 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:48.517992 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:48.518689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:48.519042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] PATH_INFO: `/v3/users/5a5f8ee0a7fd4d7a9d34e211dabab48a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:48.520303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5f8ee0a7fd4d7a9d34e211dabab48a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:48.522761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:48.523346 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e27d68ff9244f00be6bde691671affd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.535356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e27d68ff9244f00be6bde691671affd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.541653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:48.545058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e27d68ff9244f00be6bde691671affd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.545676 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e27d68ff9244f00be6bde691671affd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.553683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c7c84a27-068d-4040-9631-49247773f248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e27d68ff9244f00be6bde691671affd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.560063 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1386/2766] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:48 2022] DELETE /identity/v3/users/5e27d68ff9244f00be6bde691671affd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:48.564543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:48.564853 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:48.565098 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:48.575259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5f8ee0a7fd4d7a9d34e211dabab48a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.587159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:48.588347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:48.588963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5f8ee0a7fd4d7a9d34e211dabab48a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.589764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5f8ee0a7fd4d7a9d34e211dabab48a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.598409 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5f8ee0a7fd4d7a9d34e211dabab48a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.599130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5f8ee0a7fd4d7a9d34e211dabab48a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.615500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1381/2767] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:48 2022] DELETE /identity/v3/users/5a5f8ee0a7fd4d7a9d34e211dabab48a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:15:48.630069 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:48.653680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1387/2768] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:15:48.664047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:48.665509 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:48.730519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9822-8d2b-4db2-b8a3-411d0e224215 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:48.734850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:48.736142 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:48.736428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:48.736709 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:48.804656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:48.806814 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:48.820101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:48.821322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c7d65-490b-4585-a294-a3ee3c9a4746 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:48.821719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:48.831388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:48.832064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:48.832349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:48.832621 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] PATH_INFO: `/v3/projects/ca311e63675241eaaef42a8154e905ac` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:48.833253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1382/2769] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:48.833770 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca311e63675241eaaef42a8154e905ac)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:48.836317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:48.887999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca311e63675241eaaef42a8154e905ac', 'request_id': 'req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.893771 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca311e63675241eaaef42a8154e905ac', 'request_id': 'req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.901142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca311e63675241eaaef42a8154e905ac', 'request_id': 'req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.914728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] The token cache is being invalidate because project ca311e63675241eaaef42a8154e905ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:48.915009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.915727 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:48.917243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1388/2770] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:48 2022] DELETE /identity/v3/projects/ca311e63675241eaaef42a8154e905ac => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:48.921348 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:48.933117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5130-2d1e-46b1-b449-1c4d43c51e57 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:48.987735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:49.008685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.009441 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.009791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.010162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] PATH_INFO: `/v3/users/eb6df62737274d0da97137b2ada2d6fb` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.011421 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6df62737274d0da97137b2ada2d6fb)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.013484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.086536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6df62737274d0da97137b2ada2d6fb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.102199 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6df62737274d0da97137b2ada2d6fb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.102974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6df62737274d0da97137b2ada2d6fb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.111415 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6df62737274d0da97137b2ada2d6fb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.112180 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2389cb2a-060d-436a-bcef-0f78af42f2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6df62737274d0da97137b2ada2d6fb'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.126932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1383/2771] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:15:48 2022] DELETE /identity/v3/users/eb6df62737274d0da97137b2ada2d6fb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:49.133462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.134990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:49.150969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31731b-f5db-4bcb-b4c4-e218e48a8cfe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.213779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.214433 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.214757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.215031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] PATH_INFO: `/v3/projects/a08df817802f4957ae88ee23847c3535` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.216321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08df817802f4957ae88ee23847c3535)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.218210 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.270451 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08df817802f4957ae88ee23847c3535', 'request_id': 'req-a80c77c9-3586-469c-a0c0-073e0084ced4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.277198 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08df817802f4957ae88ee23847c3535', 'request_id': 'req-a80c77c9-3586-469c-a0c0-073e0084ced4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.283855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08df817802f4957ae88ee23847c3535', 'request_id': 'req-a80c77c9-3586-469c-a0c0-073e0084ced4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.288546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:49.300746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] The token cache is being invalidate because project a08df817802f4957ae88ee23847c3535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:49.301204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.302351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.304310 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1389/2772] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:49 2022] DELETE /identity/v3/projects/a08df817802f4957ae88ee23847c3535 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:49.311379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caa9d8-203c-4413-b1ad-cdd39a5a87d1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.311934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caa9d8-203c-4413-b1ad-cdd39a5a87d1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.312371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caa9d8-203c-4413-b1ad-cdd39a5a87d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.320876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.322262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:49.348248 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-43caa9d8-203c-4413-b1ad-cdd39a5a87d1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:49.349478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-43caa9d8-203c-4413-b1ad-cdd39a5a87d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:49.408512 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c77c9-3586-469c-a0c0-073e0084ced4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.415704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.416682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.416974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.417250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] PATH_INFO: `/v3/projects/6bd72824226c40f0bb4f52f870ec41f4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.419039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd72824226c40f0bb4f52f870ec41f4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.420992 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.428342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1384/2773] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:49.441209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d235d-226a-430b-83f9-5126bd000415 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.441496 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d235d-226a-430b-83f9-5126bd000415 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.441768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d235d-226a-430b-83f9-5126bd000415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.466560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d52d235d-226a-430b-83f9-5126bd000415 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:49.467757 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d52d235d-226a-430b-83f9-5126bd000415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:49.476352 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd72824226c40f0bb4f52f870ec41f4', 'request_id': 'req-b4d410dc-e883-4ee4-af80-c59ba258983b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.482328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd72824226c40f0bb4f52f870ec41f4', 'request_id': 'req-b4d410dc-e883-4ee4-af80-c59ba258983b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.489105 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd72824226c40f0bb4f52f870ec41f4', 'request_id': 'req-b4d410dc-e883-4ee4-af80-c59ba258983b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.508461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] The token cache is being invalidate because project 6bd72824226c40f0bb4f52f870ec41f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:15:49.509462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.509462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:15:49.511193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1390/2774] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:15:49 2022] DELETE /identity/v3/projects/6bd72824226c40f0bb4f52f870ec41f4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:49.542514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1385/2775] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:15:49.546752 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.558114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d410dc-e883-4ee4-af80-c59ba258983b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.563216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.563894 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.564153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.564421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.565073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:15:49.565375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.566993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.583880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1391/2776] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:15:49.586447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d235d-226a-430b-83f9-5126bd000415 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.592313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:49.596637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d235d-226a-430b-83f9-5126bd000415 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.604451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.605347 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.605830 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.606265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.606926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.608913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.643154 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1386/2777] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:49 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:49.648299 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.658742 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd12bf4-fd58-4db8-9340-75af4d22bab3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.738681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.739606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.740037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.740436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.741460 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.744651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.771524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1392/2778] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:49 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:49.776164 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.786725 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df2cce-d597-4499-8f92-4676f05bfc91 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.791829 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.792467 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.792742 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.793061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.793699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.795273 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.805572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1387/2779] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:15:49.809645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.814711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:49.821554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544978d-5f72-4fde-9292-4c2471aad8d5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.827325 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860723-641b-4285-aa7a-5b4dffcd2e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.828076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860723-641b-4285-aa7a-5b4dffcd2e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.828454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860723-641b-4285-aa7a-5b4dffcd2e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.828821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860723-641b-4285-aa7a-5b4dffcd2e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.829579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1860723-641b-4285-aa7a-5b4dffcd2e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.831250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1860723-641b-4285-aa7a-5b4dffcd2e80 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.843073 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1393/2780] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:49.847366 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.858185 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fcd9c-da4d-4d74-aeb9-e33bce1ab0a7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:49.862786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:49.863431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.863710 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.864024 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] PATH_INFO: `/v3/projects/0934343645484d01b9b48a40bbf0b671/users/acf06fe641e94b5b91145ed230ac7213/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.880036 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0934343645484d01b9b48a40bbf0b671, user_id=acf06fe641e94b5b91145ed230ac7213, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:49.883447 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:49.899766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1388/2781] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:49 2022] PUT /identity/v3/projects/0934343645484d01b9b48a40bbf0b671/users/acf06fe641e94b5b91145ed230ac7213/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:15:49.904777 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070104c-55a1-4c57-affb-fe739d385908 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:49.905254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070104c-55a1-4c57-affb-fe739d385908 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:49.905501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070104c-55a1-4c57-affb-fe739d385908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:49.925351 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3070104c-55a1-4c57-affb-fe739d385908 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:49.926778 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3070104c-55a1-4c57-affb-fe739d385908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:49.987032 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1394/2782] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:49.996751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:49.998478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:50.060035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea54fb-6769-4c23-aad8-2dc22d36ffa4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:50.063938 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:50.064552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:50.064833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:50.065108 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:50.075469 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:50.076974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:50.087443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1389/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:15:54.881722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33608ec7-4b20-4063-8e11-8e831b7ebdeb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:54.882387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33608ec7-4b20-4063-8e11-8e831b7ebdeb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:54.882387 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33608ec7-4b20-4063-8e11-8e831b7ebdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:54.915454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-33608ec7-4b20-4063-8e11-8e831b7ebdeb None None] MFA Rules not processed for user `acf06fe641e94b5b91145ed230ac7213`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:54.916734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-33608ec7-4b20-4063-8e11-8e831b7ebdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:55.007271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1395/2784] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:55.011876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:55.022430 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7724c65-626e-4f97-946b-bbab07df7f36 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:55.028002 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:55.028559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:55.028834 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:55.029106 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:55.029775 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:55.031669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:55.060177 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1390/2785] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:15:55 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:15:55.064427 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33608ec7-4b20-4063-8e11-8e831b7ebdeb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:55.077993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33608ec7-4b20-4063-8e11-8e831b7ebdeb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:55.142557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:55.143441 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:55.143726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:55.144040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:55.144718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:55.147768 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:55.170250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1396/2786] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:15:55 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:15:55.174365 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:55.185615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7ab30-5b69-441e-8ab2-4931c0ad331b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:55.190524 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:55.191233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:55.191512 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:55.191781 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:55.192456 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:55.194636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:55.205891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1391/2787] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:55.210681 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:55.221513 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4782aff-3eb2-4741-8384-713a070280d1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:55.227158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:55.227989 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:55.228278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:55.228559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] PATH_INFO: `/v3/projects/34ac15ce8cd2461da24ef95a72683234/users/7ae698b7b22247c79137ae6157aeae70/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:55.237762 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:55.245995 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ac15ce8cd2461da24ef95a72683234, user_id=7ae698b7b22247c79137ae6157aeae70, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:55.249171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:55.264240 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1397/2788] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:55 2022] PUT /identity/v3/projects/34ac15ce8cd2461da24ef95a72683234/users/7ae698b7b22247c79137ae6157aeae70/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:15:55.268192 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:55.279361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48cffc-e213-4199-942d-067852dd1f9a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:55.340603 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867df6cf-f670-41f7-885d-1903f11bd9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:55.341215 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df6cf-f670-41f7-885d-1903f11bd9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:55.341495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df6cf-f670-41f7-885d-1903f11bd9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:55.341765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df6cf-f670-41f7-885d-1903f11bd9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:55.342410 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867df6cf-f670-41f7-885d-1903f11bd9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:55.344098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867df6cf-f670-41f7-885d-1903f11bd9f5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:55.355857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1392/2789] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:15:55.359982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:55.376745 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62694eb6-4200-48df-8942-156b6ff0a1ce admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:55.384889 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:55.386391 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:55.386884 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:55.387412 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] PATH_INFO: `/v3/projects/34ac15ce8cd2461da24ef95a72683234/users/7ae698b7b22247c79137ae6157aeae70/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:55.389867 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ac15ce8cd2461da24ef95a72683234, user_id=7ae698b7b22247c79137ae6157aeae70, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:55.396530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:55.414570 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1398/2790] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:15:55 2022] PUT /identity/v3/projects/34ac15ce8cd2461da24ef95a72683234/users/7ae698b7b22247c79137ae6157aeae70/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:15:59.614071 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5f04e-f776-4ff4-a441-80a745424128 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:59.614572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5f04e-f776-4ff4-a441-80a745424128 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:59.614670 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5f04e-f776-4ff4-a441-80a745424128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:59.642362 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:15:59.645314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1ad5f04e-f776-4ff4-a441-80a745424128 None None] MFA Rules not processed for user `7ae698b7b22247c79137ae6157aeae70`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:15:59.647618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1ad5f04e-f776-4ff4-a441-80a745424128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:59.723144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1393/2791] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:15:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:15:59.739864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:15:59.742094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:15:59.816696 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974d324-c89d-40c3-aef7-b4b476e264b1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:15:59.822854 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:15:59.823459 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:15:59.823737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:15:59.824059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:15:59.881782 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:15:59.883378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:15:59.892397 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1399/2792] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:15:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:16:09.720262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=28 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:16:09.888300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:16:24.641771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:16:24.642324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:16:24.642324 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:16:24.661138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] MFA Rules not processed for user `acf06fe641e94b5b91145ed230ac7213`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:16:24.662253 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:16:24.669486 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1394/2793] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:16:34.656484 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:16:38.701290 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:16:38.702765 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:16:38.718336 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695681c6-37d8-44a7-ad35-d89b28e98bd0 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:16:38.724045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:16:38.724656 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:16:38.724940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:16:38.725219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:16:38.726940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:16:38.740267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:16:38.741845 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:16:38.753262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1400/2794] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:16:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:16:42.162862 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:16:42.164338 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:16:42.220179 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94c8d6-d931-4b59-981b-25453387c9cc None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:16:42.223661 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8de533-d592-42a3-97b3-12c0819650b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:16:42.224352 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de533-d592-42a3-97b3-12c0819650b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:16:42.224632 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de533-d592-42a3-97b3-12c0819650b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:16:42.224906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de533-d592-42a3-97b3-12c0819650b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:16:42.233838 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8de533-d592-42a3-97b3-12c0819650b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:16:42.235268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8de533-d592-42a3-97b3-12c0819650b8 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:16:42.244453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1395/2795] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:16:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:16:48.749390 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:16:52.240392 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=26 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:17:14.832202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:17:14.833580 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:17:14.900939 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abf081-c90d-4a0d-953f-e638c58031a2 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:17:14.905928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:17:14.906526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:17:14.906806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:17:14.907077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:17:14.909344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:17:14.920481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:17:14.922043 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:17:14.932616 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1401/2796] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:17:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:17:19.588574 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f0042-f7bc-45a2-8d6c-4921cd659c6a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:17:19.589093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f0042-f7bc-45a2-8d6c-4921cd659c6a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:17:19.589093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f0042-f7bc-45a2-8d6c-4921cd659c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:17:19.608398 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-aa7f0042-f7bc-45a2-8d6c-4921cd659c6a None None] MFA Rules not processed for user `4ad5a3ee5ab545e2893d4f6365f8f939`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:17:19.609581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aa7f0042-f7bc-45a2-8d6c-4921cd659c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:17:19.665606 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1396/2797] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:17:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:17:24.928252 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=32 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:17:29.658936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=25 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:17:30.333216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:17:30.343414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29088256-ca22-475e-8c19-460573a85b8c service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:17:30.349298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:17:30.349936 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:17:30.350223 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:17:30.350667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:17:30.352625 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:17:30.367528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:17:30.370106 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:17:30.383216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1402/2798] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:17:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 11:17:30.495214 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f0042-f7bc-45a2-8d6c-4921cd659c6a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:17:30.505233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f0042-f7bc-45a2-8d6c-4921cd659c6a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:17:30.509102 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:17:30.509712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:17:30.509990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:17:30.510481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:17:30.512429 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:17:30.528369 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:17:30.529266 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:17:30.543109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1397/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:17:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:17:34.655760 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:17:34.669756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc5254-56d0-4833-a42c-fa65d5ecd859 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:17:34.675993 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:17:34.676595 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:17:34.676870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:17:34.677142 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:17:34.678866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:17:34.691107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:17:34.692726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:17:34.703094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1403/2800] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:17:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:17:40.536667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:17:44.698722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=12 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:18:59.610118 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:18:59.620242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca21d32-f162-448e-b7a8-f77951542793 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:18:59.624774 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bc618-92be-4506-b0c5-9dbd9a6656c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:18:59.625720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bc618-92be-4506-b0c5-9dbd9a6656c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:18:59.626202 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bc618-92be-4506-b0c5-9dbd9a6656c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:18:59.626693 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bc618-92be-4506-b0c5-9dbd9a6656c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:18:59.628790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2c5bc618-92be-4506-b0c5-9dbd9a6656c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:18:59.644209 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bc618-92be-4506-b0c5-9dbd9a6656c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:18:59.647245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bc618-92be-4506-b0c5-9dbd9a6656c0 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:18:59.660160 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1398/2801] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:19:00.758414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:19:00.768273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f352ddf-fce0-4c48-aba0-9695ec119b7f service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:19:00.772901 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e082a-9f30-4825-ae2c-f45c331cefd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:19:00.773495 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e082a-9f30-4825-ae2c-f45c331cefd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:00.773775 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e082a-9f30-4825-ae2c-f45c331cefd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:00.774045 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e082a-9f30-4825-ae2c-f45c331cefd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:00.775738 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-834e082a-9f30-4825-ae2c-f45c331cefd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:00.790413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e082a-9f30-4825-ae2c-f45c331cefd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:19:00.791888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e082a-9f30-4825-ae2c-f45c331cefd3 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:19:00.805050 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1404/2802] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:19:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:19:03.947261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4145a-435a-466a-a3c3-9e68ea63d156 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:03.947751 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4145a-435a-466a-a3c3-9e68ea63d156 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:03.947951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4145a-435a-466a-a3c3-9e68ea63d156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:03.966554 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-61a4145a-435a-466a-a3c3-9e68ea63d156 None None] MFA Rules not processed for user `4ad5a3ee5ab545e2893d4f6365f8f939`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:19:03.967737 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-61a4145a-435a-466a-a3c3-9e68ea63d156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:03.975451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1399/2803] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:19:04.297015 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d6251-d61c-4ed8-8a95-734de667e150 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:04.297488 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d6251-d61c-4ed8-8a95-734de667e150 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:04.297611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d6251-d61c-4ed8-8a95-734de667e150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:04.316951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ca5d6251-d61c-4ed8-8a95-734de667e150 None None] MFA Rules not processed for user `acf06fe641e94b5b91145ed230ac7213`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:19:04.318315 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ca5d6251-d61c-4ed8-8a95-734de667e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:04.325535 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1405/2804] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:19:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:19:13.961821 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=9 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:19:14.311328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=9 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:19:23.579431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4145a-435a-466a-a3c3-9e68ea63d156 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:19:23.589157 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4145a-435a-466a-a3c3-9e68ea63d156 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:19:23.592867 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:19:23.593615 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:23.593901 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:23.594172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:23.595897 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:23.661230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:19:23.663197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:19:23.674037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1400/2805] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:19:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:19:28.597744 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d6251-d61c-4ed8-8a95-734de667e150 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:19:28.607475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d6251-d61c-4ed8-8a95-734de667e150 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:19:28.610996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:19:28.611587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:28.611892 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:28.612167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:28.613959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:28.678779 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:19:28.680338 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:19:28.689602 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1406/2806] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:19:33.669269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=28 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:19:35.265850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:19:35.267461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:35.283150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09856698-e439-45ed-98c2-138c5b1affe0 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:19:35.287623 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:19:35.288458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:35.288883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:35.289290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] PATH_INFO: `/v3/users/e00353b2edab47e79578de680e4c4d1c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:35.290423 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00353b2edab47e79578de680e4c4d1c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:19:35.292370 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:19:35.346115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00353b2edab47e79578de680e4c4d1c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.357879 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00353b2edab47e79578de680e4c4d1c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.358538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00353b2edab47e79578de680e4c4d1c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.365577 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00353b2edab47e79578de680e4c4d1c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.366929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00353b2edab47e79578de680e4c4d1c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.378768 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1401/2807] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:19:35 2022] DELETE /identity/v3/users/e00353b2edab47e79578de680e4c4d1c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:19:35.548344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:19:35.559110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fa9c6-a8b3-4d15-b9ae-2798785bbe5d service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:19:35.616533 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:19:35.617186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:35.617468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:35.617743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] PATH_INFO: `/v3/projects/9376d8470a80416cb8e57eb650884883` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:35.618862 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9376d8470a80416cb8e57eb650884883)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:19:35.620933 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:19:35.666558 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9376d8470a80416cb8e57eb650884883', 'request_id': 'req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.672211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9376d8470a80416cb8e57eb650884883', 'request_id': 'req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.679061 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9376d8470a80416cb8e57eb650884883', 'request_id': 'req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.690794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] The token cache is being invalidate because project 9376d8470a80416cb8e57eb650884883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:19:35.691085 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.691715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.693215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1407/2808] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:19:35 2022] DELETE /identity/v3/projects/9376d8470a80416cb8e57eb650884883 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:19:35.698120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:19:35.699495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:35.717098 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fea41-3da3-4c09-b61b-567e9894e723 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:19:35.774478 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:19:35.775116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:35.775391 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:35.775679 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] PATH_INFO: `/v3/users/17da95f02da34b0896ac24fbb3276359` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:35.776847 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17da95f02da34b0896ac24fbb3276359)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:19:35.778791 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:19:35.825773 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17da95f02da34b0896ac24fbb3276359'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.837656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17da95f02da34b0896ac24fbb3276359'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.838317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17da95f02da34b0896ac24fbb3276359'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.845572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17da95f02da34b0896ac24fbb3276359'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.846233 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17da95f02da34b0896ac24fbb3276359'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:35.848245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:19:35.861637 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1402/2809] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:19:35 2022] DELETE /identity/v3/users/17da95f02da34b0896ac24fbb3276359 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:19:36.033047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:19:36.034473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:19:36.058324 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39fcd4-77d2-43eb-b86d-fa10d29fca58 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:19:36.080955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:19:36.126109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:19:36.126772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:19:36.127023 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:19:36.127304 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] PATH_INFO: `/v3/projects/7b6d1cbcfb8f4dffa56b44651c1316fa` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:19:36.128444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6d1cbcfb8f4dffa56b44651c1316fa)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:19:36.130342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:19:36.178248 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6d1cbcfb8f4dffa56b44651c1316fa', 'request_id': 'req-11cba84d-c76b-42d5-b44d-a4ca5c04db86'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:36.183921 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6d1cbcfb8f4dffa56b44651c1316fa', 'request_id': 'req-11cba84d-c76b-42d5-b44d-a4ca5c04db86'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:36.190261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6d1cbcfb8f4dffa56b44651c1316fa', 'request_id': 'req-11cba84d-c76b-42d5-b44d-a4ca5c04db86'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:36.202611 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] The token cache is being invalidate because project 7b6d1cbcfb8f4dffa56b44651c1316fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:19:36.202900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:36.203551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:19:36.205064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1408/2810] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:19:36 2022] DELETE /identity/v3/projects/7b6d1cbcfb8f4dffa56b44651c1316fa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:19:45.858193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:19:46.199601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:20:34.263567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:20:34.265003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:34.323446 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f10d27-9745-4e18-b287-2131ad216990 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:20:34.370381 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba5341-9844-4122-b889-09c8ddbb9e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:20:34.370972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba5341-9844-4122-b889-09c8ddbb9e0e service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:20:34.371249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba5341-9844-4122-b889-09c8ddbb9e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:20:34.371518 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba5341-9844-4122-b889-09c8ddbb9e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:20:34.373345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-82ba5341-9844-4122-b889-09c8ddbb9e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:34.431648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba5341-9844-4122-b889-09c8ddbb9e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:20:34.433113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba5341-9844-4122-b889-09c8ddbb9e0e service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:20:34.441873 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1403/2811] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:20:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:20:44.437851 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=68 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:20:46.932096 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:20:46.934329 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:47.028221 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cba84d-c76b-42d5-b44d-a4ca5c04db86 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:20:47.034342 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:20:47.034953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:20:47.035237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:20:47.035511 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:20:47.037340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:47.054211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:20:47.056228 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:20:47.069801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1409/2812] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:20:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:20:50.467795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:20:50.468333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:20:50.468437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:20:50.487635 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] MFA Rules not processed for user `4ad5a3ee5ab545e2893d4f6365f8f939`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:20:50.488863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:50.495644 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1404/2813] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:20:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:20:50.799395 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:20:50.800927 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:50.856260 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5ce7-89a2-4dcf-9fd4-79cb8f302ff5 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:20:50.859809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:20:50.860425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:20:50.860700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:20:50.860969 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:20:50.862612 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:50.871960 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:20:50.873358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:20:50.881095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1410/2814] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:20:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:20:51.199760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:20:51.201191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:51.278041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b67a4-f278-4823-a090-bc47239e6f96 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:20:51.283015 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:20:51.283654 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:20:51.283967 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:20:51.284242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:20:51.285951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:51.297903 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:20:51.299522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:20:51.309534 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1405/2815] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:20:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:20:53.163046 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:20:53.164667 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:53.175825 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3386a-1802-4068-a7b4-5c908e122686 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:20:53.179592 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faa568-7764-465d-823a-673323ed388d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:20:53.180236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faa568-7764-465d-823a-673323ed388d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:20:53.180516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faa568-7764-465d-823a-673323ed388d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:20:53.180785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faa568-7764-465d-823a-673323ed388d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:20:53.182704 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-15faa568-7764-465d-823a-673323ed388d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:20:53.193277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faa568-7764-465d-823a-673323ed388d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:20:53.194713 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faa568-7764-465d-823a-673323ed388d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:20:53.203863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1411/2816] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:20:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:21:01.305159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=35 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:21:03.199913 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=66 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:21:28.804332 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:21:28.805719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:21:28.817963 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9602-cd82-415d-9840-d67a37277e95 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:21:28.821491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:21:28.822100 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:21:28.822376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:21:28.822646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:21:28.824341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:21:28.885068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:21:28.886513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:21:28.895707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1406/2817] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:21:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:21:38.891671 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=28 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:21:39.909354 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064db2c-41da-4432-a9e6-cd7c2b54d32a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:21:39.909897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064db2c-41da-4432-a9e6-cd7c2b54d32a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:21:39.909897 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064db2c-41da-4432-a9e6-cd7c2b54d32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:21:39.929349 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0064db2c-41da-4432-a9e6-cd7c2b54d32a None None] MFA Rules not processed for user `acf06fe641e94b5b91145ed230ac7213`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:21:39.930526 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0064db2c-41da-4432-a9e6-cd7c2b54d32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:21:39.937204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1412/2818] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:21:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:21:49.924259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:21:54.492481 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:21:54.495125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:21:54.513822 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25561b09-c07e-4ff0-8fc1-e26a4484189b service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:21:54.520374 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:21:54.520970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:21:54.521251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:21:54.521522 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:21:54.523538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:21:54.536041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:21:54.537626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:21:54.548628 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1407/2819] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:21:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:21:57.433375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064db2c-41da-4432-a9e6-cd7c2b54d32a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:21:57.444006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064db2c-41da-4432-a9e6-cd7c2b54d32a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:21:57.447821 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:21:57.448445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:21:57.448724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:21:57.448998 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:21:57.450700 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:21:57.512857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:21:57.514273 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:21:57.523802 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1413/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:21:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:04.543720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:07.519335 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=30 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:16.127668 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:16.129719 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:16.212624 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448d6a1-9371-44c3-8783-82c2cd1e59ca service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:16.217747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:16.218345 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:16.218626 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:16.218899 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:16.220640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:16.232923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:16.234536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:16.245259 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1408/2821] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:26.240924 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=37 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:41.541776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:41.559919 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81566964-bb37-4759-8d8e-45410eda2cb5 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:41.563583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:41.564203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:41.564481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:41.564754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:41.566468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:41.650236 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:41.651588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:41.670678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1414/2822] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:44.488787 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:44.514445 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ab32-82a2-4f8d-8133-2445fdb74b06 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:44.524525 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:44.525149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:44.525428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:44.525700 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:44.527601 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:44.541712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:44.543459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:44.564460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1409/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:44.576978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:44.580047 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:44.593366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8327e3-ece7-4a6f-92a0-93e6fa965b7e service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:44.600489 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:44.601144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:44.601442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:44.601710 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] PATH_INFO: `/v3/users/4ad5a3ee5ab545e2893d4f6365f8f939` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:44.602890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad5a3ee5ab545e2893d4f6365f8f939)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:44.604886 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:44.652944 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad5a3ee5ab545e2893d4f6365f8f939'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:44.664937 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad5a3ee5ab545e2893d4f6365f8f939'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:44.672546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad5a3ee5ab545e2893d4f6365f8f939'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:44.673250 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad5a3ee5ab545e2893d4f6365f8f939'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:44.681849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-764cc54d-1bdc-4ac6-92b5-2599e2b0f1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad5a3ee5ab545e2893d4f6365f8f939'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:44.687860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1415/2824] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:22:44 2022] DELETE /identity/v3/users/4ad5a3ee5ab545e2893d4f6365f8f939 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:44.858379 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:44.869618 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275dc95-b0b5-4392-8a71-527f5b4f972f service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:44.931720 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:44.932367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:44.932645 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:44.932915 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] PATH_INFO: `/v3/projects/5d69675576dc4c7fa5cede0f99db8e32` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:44.934019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d69675576dc4c7fa5cede0f99db8e32)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:44.935929 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:44.982917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d69675576dc4c7fa5cede0f99db8e32', 'request_id': 'req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:44.988313 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d69675576dc4c7fa5cede0f99db8e32', 'request_id': 'req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:44.994475 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d69675576dc4c7fa5cede0f99db8e32', 'request_id': 'req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:45.006837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] The token cache is being invalidate because project 5d69675576dc4c7fa5cede0f99db8e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:22:45.007121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:45.007733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-550c057b-6e47-40f2-94d4-5eaaa6c17f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:45.009223 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1410/2825] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:22:44 2022] DELETE /identity/v3/projects/5d69675576dc4c7fa5cede0f99db8e32 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:22:45.014442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5b664-1f1a-4176-8ae2-4bddbc75b2f9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.014729 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5b664-1f1a-4176-8ae2-4bddbc75b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.015000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5b664-1f1a-4176-8ae2-4bddbc75b2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.059506 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:45.081829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-edf5b664-1f1a-4176-8ae2-4bddbc75b2f9 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:45.083133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-edf5b664-1f1a-4176-8ae2-4bddbc75b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:45.140761 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1416/2826] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:45.149746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d98f7-f5cd-49a0-85bb-00d79c2b621a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.150137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d98f7-f5cd-49a0-85bb-00d79c2b621a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.150485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d98f7-f5cd-49a0-85bb-00d79c2b621a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.170870 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f86d98f7-f5cd-49a0-85bb-00d79c2b621a None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:45.172130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f86d98f7-f5cd-49a0-85bb-00d79c2b621a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:45.181291 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1411/2827] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:45.185454 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5b664-1f1a-4176-8ae2-4bddbc75b2f9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:45.195988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5b664-1f1a-4176-8ae2-4bddbc75b2f9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:45.200608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:45.201225 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.201504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.201776 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.202417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:22:45.202719 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:45.204355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:45.216584 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1417/2828] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:22:45.221115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d98f7-f5cd-49a0-85bb-00d79c2b621a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:45.230536 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d98f7-f5cd-49a0-85bb-00d79c2b621a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:45.235138 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:45.235717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.236052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.236336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.236951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:45.239993 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:45.266428 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1412/2829] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:22:45 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:45.270695 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:45.280943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642548-8aa3-4e19-84e5-b2d3c05cdd04 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:45.341339 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:45.341863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.342144 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.342414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.343042 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:45.344981 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:45.367216 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1418/2830] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:22:45 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:45.371706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:45.382703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5850c-d1c6-4dfd-9f35-30e566f85d7c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:45.387418 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:45.388043 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.388326 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.388599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.389249 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:45.391271 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:45.401318 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1413/2831] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:45.405302 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:45.415214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd840-023e-48c9-a910-a32ca9bfd620 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:45.419678 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cff18-6a06-4434-824c-85616682dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:45.420301 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cff18-6a06-4434-824c-85616682dc82 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.420579 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cff18-6a06-4434-824c-85616682dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.420851 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cff18-6a06-4434-824c-85616682dc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.421484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cff18-6a06-4434-824c-85616682dc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:45.423058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cff18-6a06-4434-824c-85616682dc82 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:45.435125 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1419/2832] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:45.439385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:45.450145 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66df4b-156f-4a1d-ac41-0c1b7db67d92 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:45.454695 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:45.455331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.455612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.456041 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] PATH_INFO: `/v3/projects/672e8372532f48379b67b4ba4609a351/users/d19ef299f7ef495c83681e624fa0c314/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.470683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672e8372532f48379b67b4ba4609a351, user_id=d19ef299f7ef495c83681e624fa0c314, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:45.474096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:45.486578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1414/2833] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:22:45 2022] PUT /identity/v3/projects/672e8372532f48379b67b4ba4609a351/users/d19ef299f7ef495c83681e624fa0c314/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:45.497522 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc1cc1-5e37-4701-be7e-c64ca62d0b07 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.497803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc1cc1-5e37-4701-be7e-c64ca62d0b07 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.498072 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc1cc1-5e37-4701-be7e-c64ca62d0b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.525276 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-83fc1cc1-5e37-4701-be7e-c64ca62d0b07 None None] MFA Rules not processed for user `d19ef299f7ef495c83681e624fa0c314`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:45.526463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-83fc1cc1-5e37-4701-be7e-c64ca62d0b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:45.572754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:45.590593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1420/2834] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:45.603113 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:45.604893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:45.620244 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:45.681502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394021-6ca6-4557-9795-7cee76bbaaf2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:45.687044 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5d689-b746-45eb-b14c-7ee2ccc1ec69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:45.687636 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5d689-b746-45eb-b14c-7ee2ccc1ec69 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:45.687947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5d689-b746-45eb-b14c-7ee2ccc1ec69 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:45.688218 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5d689-b746-45eb-b14c-7ee2ccc1ec69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:45.696130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5d689-b746-45eb-b14c-7ee2ccc1ec69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:45.697698 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5d689-b746-45eb-b14c-7ee2ccc1ec69 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:45.706251 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1415/2835] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:55.075340 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc1cc1-5e37-4701-be7e-c64ca62d0b07 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:55.134219 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc1cc1-5e37-4701-be7e-c64ca62d0b07 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:55.138658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:55.139282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.139565 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.139859 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] PATH_INFO: `/v3/users/d19ef299f7ef495c83681e624fa0c314` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.140990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d19ef299f7ef495c83681e624fa0c314)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:55.142905 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:55.190159 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd19ef299f7ef495c83681e624fa0c314'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.202574 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd19ef299f7ef495c83681e624fa0c314'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.210215 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd19ef299f7ef495c83681e624fa0c314'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.210860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd19ef299f7ef495c83681e624fa0c314'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.218039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd19ef299f7ef495c83681e624fa0c314'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.224405 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1421/2836] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:22:55 2022] DELETE /identity/v3/users/d19ef299f7ef495c83681e624fa0c314 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:55.228923 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.229434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.229434 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.248150 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:55.249337 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:55.307893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1416/2837] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:55.318286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:55.319686 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:55.376462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78cc3e-5cd7-444d-8228-e89992725f3b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:55.380734 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4620b508-4a76-41eb-b77b-696e1f092945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:55.381345 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4620b508-4a76-41eb-b77b-696e1f092945 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.381620 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4620b508-4a76-41eb-b77b-696e1f092945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.381888 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4620b508-4a76-41eb-b77b-696e1f092945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.390963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4620b508-4a76-41eb-b77b-696e1f092945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:55.392374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4620b508-4a76-41eb-b77b-696e1f092945 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:55.403006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1422/2838] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:55.672084 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:55.673505 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:55.686146 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0efdf0-d8a8-4598-b4df-83172364f58f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:55.690812 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:55.691436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.691712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.692007 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] PATH_INFO: `/v3/projects/672e8372532f48379b67b4ba4609a351` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.693872 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672e8372532f48379b67b4ba4609a351)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:55.695767 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:55.742093 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672e8372532f48379b67b4ba4609a351', 'request_id': 'req-a269a0a9-807d-441a-85e3-0fc30435e3c7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.747949 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672e8372532f48379b67b4ba4609a351', 'request_id': 'req-a269a0a9-807d-441a-85e3-0fc30435e3c7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.754350 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672e8372532f48379b67b4ba4609a351', 'request_id': 'req-a269a0a9-807d-441a-85e3-0fc30435e3c7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.767236 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] The token cache is being invalidate because project 672e8372532f48379b67b4ba4609a351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:22:55.767526 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.768166 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-a269a0a9-807d-441a-85e3-0fc30435e3c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:55.769669 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1417/2839] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:22:55 2022] DELETE /identity/v3/projects/672e8372532f48379b67b4ba4609a351 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:55.774878 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3466-bd56-41a3-bda3-5da7c319ac59 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.775186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3466-bd56-41a3-bda3-5da7c319ac59 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.775462 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3466-bd56-41a3-bda3-5da7c319ac59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.794757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ede3466-bd56-41a3-bda3-5da7c319ac59 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:55.796004 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ede3466-bd56-41a3-bda3-5da7c319ac59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:55.805078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:55.858640 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1423/2840] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:22:55.867197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf658317-96d3-4926-b17f-90f8096bc48f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.867487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf658317-96d3-4926-b17f-90f8096bc48f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.867766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf658317-96d3-4926-b17f-90f8096bc48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.887746 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bf658317-96d3-4926-b17f-90f8096bc48f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:55.889149 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bf658317-96d3-4926-b17f-90f8096bc48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:55.897317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1418/2841] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:55.901543 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede3466-bd56-41a3-bda3-5da7c319ac59 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:55.911139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede3466-bd56-41a3-bda3-5da7c319ac59 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:55.916453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:55.916988 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.917267 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.917539 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.918180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:22:55.918481 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:55.920093 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:55.932174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1424/2842] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:22:55.936275 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf658317-96d3-4926-b17f-90f8096bc48f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:55.945513 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf658317-96d3-4926-b17f-90f8096bc48f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:55.950190 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:55.950782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:55.951062 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:55.951336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:55.952003 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:55.953893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:55.968592 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:55.983094 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1419/2843] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:22:55 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:55.987287 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:55.998044 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ac4a6-091f-4a67-a608-14434c1a6bd1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.055608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.056243 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.056524 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.056798 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.057456 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.059551 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.080763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1425/2844] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:22:55 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:56.084892 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.095203 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e54f9-419d-4c34-854f-885db2980793 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.100535 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.101121 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.101404 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.101678 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.102315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.103931 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.113855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1420/2845] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:56.118394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.128133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b538598-cf11-47fc-a748-c385cd23cda5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.132817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab6a17-9eb1-4f6a-8100-9b70fff0e6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.133403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab6a17-9eb1-4f6a-8100-9b70fff0e6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.133680 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab6a17-9eb1-4f6a-8100-9b70fff0e6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.133950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab6a17-9eb1-4f6a-8100-9b70fff0e6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.134583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab6a17-9eb1-4f6a-8100-9b70fff0e6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.136187 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab6a17-9eb1-4f6a-8100-9b70fff0e6bf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.147123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1426/2846] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:56.151170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.162072 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26d7b-e2b3-4f23-b03c-e85745333749 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.166806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.167371 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.167652 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.167960 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] PATH_INFO: `/v3/projects/0913ade92f8c4872931a1f72dde83a04/users/41b9be4520c04c359792e7add08330c8/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.183913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0913ade92f8c4872931a1f72dde83a04, user_id=41b9be4520c04c359792e7add08330c8, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.186555 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.199511 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1421/2847] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:22:56 2022] PUT /identity/v3/projects/0913ade92f8c4872931a1f72dde83a04/users/41b9be4520c04c359792e7add08330c8/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:56.211107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3e70-491d-481b-96c3-9adacf6862ce None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.211396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3e70-491d-481b-96c3-9adacf6862ce None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.211671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3e70-491d-481b-96c3-9adacf6862ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.239311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f89b3e70-491d-481b-96c3-9adacf6862ce None None] MFA Rules not processed for user `41b9be4520c04c359792e7add08330c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:56.241193 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f89b3e70-491d-481b-96c3-9adacf6862ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:56.303971 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1427/2848] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:56.308493 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.370221 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f533b-bdbd-4d34-89b5-f9a6fa579e4d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.374795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.375380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.375656 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.375957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.376584 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.378463 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.408014 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1422/2849] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:22:56 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 11:22:56.412833 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b3e70-491d-481b-96c3-9adacf6862ce None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.424692 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b3e70-491d-481b-96c3-9adacf6862ce None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.429204 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:56.482654 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.483254 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.483534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.483808 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.484473 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.486373 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.506403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1428/2850] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:22:56 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:56.510464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.521197 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957568-f66a-4d69-bbe7-2033a8d7ae10 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.526116 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.526712 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.526990 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.527262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.527934 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.529521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.540569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1423/2851] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:56.544683 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.555064 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dce085-5c4c-4640-bd04-038032ca3741 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.559661 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.560333 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.560797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.561087 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] PATH_INFO: `/v3/projects/1852e3b85c9441728486eb24bd18d145/users/966cd0ba3ed74183a737e8f7aaf7983a/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.578866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1852e3b85c9441728486eb24bd18d145, user_id=966cd0ba3ed74183a737e8f7aaf7983a, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.581642 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.595009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1429/2852] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:22:56 2022] PUT /identity/v3/projects/1852e3b85c9441728486eb24bd18d145/users/966cd0ba3ed74183a737e8f7aaf7983a/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:22:56.599595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.609483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b7807-964d-4748-a46b-7a25f29ffabc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.665806 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3101516-dfc2-4c7f-b392-87bff7d149ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.666399 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3101516-dfc2-4c7f-b392-87bff7d149ee admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.666680 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3101516-dfc2-4c7f-b392-87bff7d149ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.666972 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3101516-dfc2-4c7f-b392-87bff7d149ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.667605 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3101516-dfc2-4c7f-b392-87bff7d149ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.669226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3101516-dfc2-4c7f-b392-87bff7d149ee admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.678891 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1424/2853] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:56.683000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.693881 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8d8d9-ccb2-4f60-8b3d-b61af2927636 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.698499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988da235-0efa-4557-a454-c3556136213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.699139 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da235-0efa-4557-a454-c3556136213f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.699423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da235-0efa-4557-a454-c3556136213f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.699699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da235-0efa-4557-a454-c3556136213f admin admin] PATH_INFO: `/v3/projects/1852e3b85c9441728486eb24bd18d145/users/966cd0ba3ed74183a737e8f7aaf7983a/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.701651 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988da235-0efa-4557-a454-c3556136213f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1852e3b85c9441728486eb24bd18d145, user_id=966cd0ba3ed74183a737e8f7aaf7983a, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.704358 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988da235-0efa-4557-a454-c3556136213f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.715239 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1430/2854] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:22:56 2022] PUT /identity/v3/projects/1852e3b85c9441728486eb24bd18d145/users/966cd0ba3ed74183a737e8f7aaf7983a/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:56.725658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b836d6-c041-46bf-957d-c5ebd6303b7f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.725944 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b836d6-c041-46bf-957d-c5ebd6303b7f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.726216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b836d6-c041-46bf-957d-c5ebd6303b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.753240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e8b836d6-c041-46bf-957d-c5ebd6303b7f None None] MFA Rules not processed for user `966cd0ba3ed74183a737e8f7aaf7983a`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:56.754422 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e8b836d6-c041-46bf-957d-c5ebd6303b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:56.772303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:56.823760 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1425/2855] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:56.834950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988da235-0efa-4557-a454-c3556136213f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:56.836672 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-988da235-0efa-4557-a454-c3556136213f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:56.892715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988da235-0efa-4557-a454-c3556136213f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:56.897575 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d544e-844b-4680-917c-967124fca9ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:56.898174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d544e-844b-4680-917c-967124fca9ac service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:56.898452 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d544e-844b-4680-917c-967124fca9ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:56.898724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d544e-844b-4680-917c-967124fca9ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:56.907820 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d544e-844b-4680-917c-967124fca9ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:56.909277 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d544e-844b-4680-917c-967124fca9ac service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:56.918665 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1431/2856] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:57.065115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b836d6-c041-46bf-957d-c5ebd6303b7f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:57.127323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b836d6-c041-46bf-957d-c5ebd6303b7f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:57.132682 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:57.133309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:57.133587 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:57.133854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] PATH_INFO: `/v3/users/41b9be4520c04c359792e7add08330c8` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:57.134966 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b9be4520c04c359792e7add08330c8)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:57.137022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:57.185378 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b9be4520c04c359792e7add08330c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.197886 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b9be4520c04c359792e7add08330c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.198487 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b9be4520c04c359792e7add08330c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.206242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b9be4520c04c359792e7add08330c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.206813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b9be4520c04c359792e7add08330c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.218893 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1426/2857] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:22:57 2022] DELETE /identity/v3/users/41b9be4520c04c359792e7add08330c8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:57.223554 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:57.224163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:57.224307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:57.243230 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:57.244455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:57.301751 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1432/2858] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:57.312022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:57.313474 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:57.349761 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:57.373115 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc227-213c-4237-8fda-7efcfa2e23af admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:57.376703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:57.377314 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:57.377570 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:57.377839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:57.387262 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:57.388731 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:57.398912 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1427/2859] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:22:57.662214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:57.663688 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:57.672458 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:57.676756 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a1ed4-56a9-46e3-b008-2e5349b2cd01 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:57.684923 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:57.685563 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:57.685840 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:57.686109 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] PATH_INFO: `/v3/projects/0913ade92f8c4872931a1f72dde83a04` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:57.687282 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0913ade92f8c4872931a1f72dde83a04)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:57.689157 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:57.736275 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0913ade92f8c4872931a1f72dde83a04', 'request_id': 'req-9f5dac80-a409-4659-916b-e74ce2878770'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.742450 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0913ade92f8c4872931a1f72dde83a04', 'request_id': 'req-9f5dac80-a409-4659-916b-e74ce2878770'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.748593 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0913ade92f8c4872931a1f72dde83a04', 'request_id': 'req-9f5dac80-a409-4659-916b-e74ce2878770'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.760557 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] The token cache is being invalidate because project 0913ade92f8c4872931a1f72dde83a04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:22:57.760839 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.761453 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.762914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1433/2860] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:22:57 2022] DELETE /identity/v3/projects/0913ade92f8c4872931a1f72dde83a04 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:57.766894 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:57.777335 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674a7d-5b20-400b-ba3c-774a6a363d55 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:57.832235 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:57.832854 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:57.833663 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:57.834082 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] PATH_INFO: `/v3/users/966cd0ba3ed74183a737e8f7aaf7983a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:57.835208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=966cd0ba3ed74183a737e8f7aaf7983a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:57.837206 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:57.881887 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966cd0ba3ed74183a737e8f7aaf7983a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.893356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966cd0ba3ed74183a737e8f7aaf7983a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.893999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966cd0ba3ed74183a737e8f7aaf7983a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.901837 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966cd0ba3ed74183a737e8f7aaf7983a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.902436 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5aa4f8a1-011f-4629-846e-0c63b7875a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966cd0ba3ed74183a737e8f7aaf7983a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:57.914930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1428/2861] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:22:57 2022] DELETE /identity/v3/users/966cd0ba3ed74183a737e8f7aaf7983a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:58.418671 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:58.420263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:58.433682 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dac80-a409-4659-916b-e74ce2878770 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:58.491493 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:58.492150 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.492423 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.492694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] PATH_INFO: `/v3/projects/1852e3b85c9441728486eb24bd18d145` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.493790 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1852e3b85c9441728486eb24bd18d145)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:58.495679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:58.541289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1852e3b85c9441728486eb24bd18d145', 'request_id': 'req-75e74c78-97d4-478a-bc86-8108313a1875'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:58.546732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1852e3b85c9441728486eb24bd18d145', 'request_id': 'req-75e74c78-97d4-478a-bc86-8108313a1875'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:58.553784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1852e3b85c9441728486eb24bd18d145', 'request_id': 'req-75e74c78-97d4-478a-bc86-8108313a1875'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:58.565031 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] The token cache is being invalidate because project 1852e3b85c9441728486eb24bd18d145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:22:58.565321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:58.565946 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75e74c78-97d4-478a-bc86-8108313a1875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:22:58.567407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1434/2862] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:22:58 2022] DELETE /identity/v3/projects/1852e3b85c9441728486eb24bd18d145 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:58.573807 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38dd8e-ab51-41ef-9cee-68c9bf47a464 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.574300 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38dd8e-ab51-41ef-9cee-68c9bf47a464 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.574417 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38dd8e-ab51-41ef-9cee-68c9bf47a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.594183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-be38dd8e-ab51-41ef-9cee-68c9bf47a464 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:58.595380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-be38dd8e-ab51-41ef-9cee-68c9bf47a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:58.644759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:58.658395 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1429/2863] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:58.666037 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b23fe-fc73-4650-92ec-db1cdac324db None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.666327 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b23fe-fc73-4650-92ec-db1cdac324db None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.666608 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b23fe-fc73-4650-92ec-db1cdac324db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.686753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f30b23fe-fc73-4650-92ec-db1cdac324db None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:58.687997 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f30b23fe-fc73-4650-92ec-db1cdac324db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:58.696722 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1435/2864] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:58.700833 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38dd8e-ab51-41ef-9cee-68c9bf47a464 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:58.710675 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38dd8e-ab51-41ef-9cee-68c9bf47a464 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:58.715274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:58.715882 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.716168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.716438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.717079 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:22:58.717380 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:58.718997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:58.730686 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1430/2865] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 11:22:58.734982 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b23fe-fc73-4650-92ec-db1cdac324db None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:58.744834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b23fe-fc73-4650-92ec-db1cdac324db None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:58.749497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:58.750090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.750370 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.750645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.751286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:58.753222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:58.763347 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:58.782009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1436/2866] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:22:58 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:58.786178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:58.796407 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66bc8f-a26f-4584-918a-75bf4290ceb0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:58.851952 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:58.852553 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.852835 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.853109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.853726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:58.855641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:58.877274 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1431/2867] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:22:58 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:58.882226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:58.892769 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837fd3b-4178-4877-98fe-9c0cc1701259 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:58.898813 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:58.899637 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.899953 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.900262 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.901143 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:58.903379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:58.914720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1437/2868] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:58.918747 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:58.929170 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da9830-096a-411d-aa28-38ce9482aa21 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:58.933674 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263dc65-4783-473f-a1e5-6075dd0be9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:58.934255 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263dc65-4783-473f-a1e5-6075dd0be9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.934530 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263dc65-4783-473f-a1e5-6075dd0be9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.934800 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263dc65-4783-473f-a1e5-6075dd0be9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.935458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263dc65-4783-473f-a1e5-6075dd0be9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:58.937068 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263dc65-4783-473f-a1e5-6075dd0be9fa admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:58.947646 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1432/2869] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:58.951724 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:58.962077 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546ef03-dcbe-49e7-8493-ef267462305d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:58.966800 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:58.967437 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:58.967715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:58.968029 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] PATH_INFO: `/v3/projects/2657aec51ba640fba15634ee987f22fe/users/1635f46ee07d471186e07c22a55c1ed6/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:58.982707 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2657aec51ba640fba15634ee987f22fe, user_id=1635f46ee07d471186e07c22a55c1ed6, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:58.985379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:58.998548 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1438/2870] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:22:58 2022] PUT /identity/v3/projects/2657aec51ba640fba15634ee987f22fe/users/1635f46ee07d471186e07c22a55c1ed6/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:59.062373 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432216c-7214-4cf2-a6a9-3034db89a6d9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.062655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432216c-7214-4cf2-a6a9-3034db89a6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.062930 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432216c-7214-4cf2-a6a9-3034db89a6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.089797 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0432216c-7214-4cf2-a6a9-3034db89a6d9 None None] MFA Rules not processed for user `1635f46ee07d471186e07c22a55c1ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:59.090975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0432216c-7214-4cf2-a6a9-3034db89a6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:59.154351 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1433/2871] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:59.159365 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:59.221817 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a20de9-2769-4162-a9e5-5258f836faa7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:59.226999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:59.227596 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.227898 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.228174 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.228824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:59.230715 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:59.257220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1439/2872] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:22:59 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:59.261357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432216c-7214-4cf2-a6a9-3034db89a6d9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:59.272689 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432216c-7214-4cf2-a6a9-3034db89a6d9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:59.318586 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:59.329691 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:59.330316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.330591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.330861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.331477 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:59.334057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:59.354559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1434/2873] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:22:59 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:22:59.358544 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:59.369071 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae4969-777e-49aa-b53a-53df1814b828 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:59.373815 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:59.374415 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.374694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.374990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.375631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:59.377251 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:59.388601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1440/2874] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:59.392766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:59.404120 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd13d-4bfd-41eb-a660-db62b8403c15 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:59.408727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:59.409316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.409590 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.409861 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] PATH_INFO: `/v3/projects/194d800c994740bd957d356707b929ac/users/25f3fc65a6d0474a9666e7d2a1244c73/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.425270 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194d800c994740bd957d356707b929ac, user_id=25f3fc65a6d0474a9666e7d2a1244c73, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:59.428186 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:59.440730 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1435/2875] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:22:59 2022] PUT /identity/v3/projects/194d800c994740bd957d356707b929ac/users/25f3fc65a6d0474a9666e7d2a1244c73/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:59.445101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:59.455220 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a075a-7fb5-4dd9-8b5b-f29c3872e3bd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:59.511470 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33179d8-6135-482e-a4ca-3be43b680afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:59.512095 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33179d8-6135-482e-a4ca-3be43b680afc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.512382 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33179d8-6135-482e-a4ca-3be43b680afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.512655 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33179d8-6135-482e-a4ca-3be43b680afc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.513286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33179d8-6135-482e-a4ca-3be43b680afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:59.514864 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33179d8-6135-482e-a4ca-3be43b680afc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:59.525203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1441/2876] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:22:59.529460 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:59.540926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7b36d-7b38-49e9-9a7b-95ebfd9b35d3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:59.545547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:59.546213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.546491 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.546764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] PATH_INFO: `/v3/projects/194d800c994740bd957d356707b929ac/users/25f3fc65a6d0474a9666e7d2a1244c73/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.548732 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=194d800c994740bd957d356707b929ac, user_id=25f3fc65a6d0474a9666e7d2a1244c73, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:59.551356 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:59.561480 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1436/2877] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:22:59 2022] PUT /identity/v3/projects/194d800c994740bd957d356707b929ac/users/25f3fc65a6d0474a9666e7d2a1244c73/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:22:59.571375 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdba23-361f-4b38-a159-97bf9ecd8f91 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.571660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdba23-361f-4b38-a159-97bf9ecd8f91 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.571968 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdba23-361f-4b38-a159-97bf9ecd8f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.600378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7acdba23-361f-4b38-a159-97bf9ecd8f91 None None] MFA Rules not processed for user `25f3fc65a6d0474a9666e7d2a1244c73`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:22:59.601585 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7acdba23-361f-4b38-a159-97bf9ecd8f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:59.613153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:22:59.670261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1442/2878] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:22:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:22:59.681027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:22:59.682703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:22:59.742500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a4e61-6402-4155-a007-a2123fb151cb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:22:59.746066 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7e0c2-959a-4942-b0da-cda139c52f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:22:59.746683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e0c2-959a-4942-b0da-cda139c52f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:22:59.746957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e0c2-959a-4942-b0da-cda139c52f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:22:59.747225 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e0c2-959a-4942-b0da-cda139c52f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:22:59.756443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7e0c2-959a-4942-b0da-cda139c52f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:22:59.757863 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7e0c2-959a-4942-b0da-cda139c52f92 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:22:59.767012 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1437/2879] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:22:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:00.136100 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdba23-361f-4b38-a159-97bf9ecd8f91 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:00.200067 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdba23-361f-4b38-a159-97bf9ecd8f91 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:00.204497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:00.205116 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:00.205394 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:00.205663 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] PATH_INFO: `/v3/users/1635f46ee07d471186e07c22a55c1ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:00.206757 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1635f46ee07d471186e07c22a55c1ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:00.208699 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:00.253916 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1635f46ee07d471186e07c22a55c1ed6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.266633 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1635f46ee07d471186e07c22a55c1ed6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.274185 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1635f46ee07d471186e07c22a55c1ed6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.274996 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1635f46ee07d471186e07c22a55c1ed6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.281872 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1635f46ee07d471186e07c22a55c1ed6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.288259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1443/2880] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:23:00 2022] DELETE /identity/v3/users/1635f46ee07d471186e07c22a55c1ed6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:00.292880 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:00.293168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:00.293443 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:00.313054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:00.314242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:00.373228 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1438/2881] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:00.383516 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:00.384932 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:00.417289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:00.444572 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b19519-96c4-42a0-8a37-cee58ba08f4a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:00.448195 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:00.449058 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:00.449534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:00.450016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:00.463114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:00.464514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:00.474317 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1444/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:00.743913 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:00.745464 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:00.758086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb26df-d9f8-491c-9ac7-8f4eb96b8331 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:00.762971 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:00.763604 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:00.763917 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:00.764195 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] PATH_INFO: `/v3/projects/2657aec51ba640fba15634ee987f22fe` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:00.765308 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2657aec51ba640fba15634ee987f22fe)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:00.767213 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:00.810219 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:00.817167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2657aec51ba640fba15634ee987f22fe', 'request_id': 'req-491cde95-c351-47e0-b654-fdb3082eed01'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.823268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2657aec51ba640fba15634ee987f22fe', 'request_id': 'req-491cde95-c351-47e0-b654-fdb3082eed01'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.829569 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2657aec51ba640fba15634ee987f22fe', 'request_id': 'req-491cde95-c351-47e0-b654-fdb3082eed01'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.842125 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] The token cache is being invalidate because project 2657aec51ba640fba15634ee987f22fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:23:00.842411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.843031 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.844540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1439/2883] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:23:00 2022] DELETE /identity/v3/projects/2657aec51ba640fba15634ee987f22fe => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:00.848540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:00.859147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d720a5-6851-4a3c-9e3b-7aecb3e4095c service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:00.914674 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:00.915307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:00.915589 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:00.915900 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] PATH_INFO: `/v3/users/25f3fc65a6d0474a9666e7d2a1244c73` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:00.917025 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f3fc65a6d0474a9666e7d2a1244c73)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:00.918935 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:00.963573 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f3fc65a6d0474a9666e7d2a1244c73'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.975755 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f3fc65a6d0474a9666e7d2a1244c73'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.983300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f3fc65a6d0474a9666e7d2a1244c73'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.983980 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f3fc65a6d0474a9666e7d2a1244c73'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.990870 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-915a435f-5f30-466c-bd6e-c31f780e0368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f3fc65a6d0474a9666e7d2a1244c73'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:00.996569 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1445/2884] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:23:00 2022] DELETE /identity/v3/users/25f3fc65a6d0474a9666e7d2a1244c73 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:01.279022 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:01.280547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:01.293683 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cde95-c351-47e0-b654-fdb3082eed01 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:01.350252 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:01.350877 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.351155 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.351426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] PATH_INFO: `/v3/projects/194d800c994740bd957d356707b929ac` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.352550 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=194d800c994740bd957d356707b929ac)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:01.354444 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:01.401588 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194d800c994740bd957d356707b929ac', 'request_id': 'req-87dc053d-4c6a-47aa-a152-4d60fe9239ea'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:01.406970 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194d800c994740bd957d356707b929ac', 'request_id': 'req-87dc053d-4c6a-47aa-a152-4d60fe9239ea'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:01.413437 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194d800c994740bd957d356707b929ac', 'request_id': 'req-87dc053d-4c6a-47aa-a152-4d60fe9239ea'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:01.425483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] The token cache is being invalidate because project 194d800c994740bd957d356707b929ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:23:01.425765 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:01.426383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-87dc053d-4c6a-47aa-a152-4d60fe9239ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:01.427914 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1440/2885] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:23:01 2022] DELETE /identity/v3/projects/194d800c994740bd957d356707b929ac => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:01.432721 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b280be0-57e7-4fce-a858-1dff6f7db853 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.433009 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b280be0-57e7-4fce-a858-1dff6f7db853 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.433286 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b280be0-57e7-4fce-a858-1dff6f7db853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.452822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3b280be0-57e7-4fce-a858-1dff6f7db853 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:01.453926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3b280be0-57e7-4fce-a858-1dff6f7db853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:01.497366 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:01.513039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1446/2886] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:01.524936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082a2b-71c6-4562-80da-37c58a33ec62 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.525703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082a2b-71c6-4562-80da-37c58a33ec62 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.526049 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082a2b-71c6-4562-80da-37c58a33ec62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.550458 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fd082a2b-71c6-4562-80da-37c58a33ec62 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:01.551649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fd082a2b-71c6-4562-80da-37c58a33ec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:01.560187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1441/2887] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:01.564652 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b280be0-57e7-4fce-a858-1dff6f7db853 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:01.575376 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b280be0-57e7-4fce-a858-1dff6f7db853 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:01.580034 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:01.580632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.580914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.581189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.581831 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:23:01.582133 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:01.583754 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:01.595131 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1447/2888] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:23:01.599500 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082a2b-71c6-4562-80da-37c58a33ec62 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:01.608782 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082a2b-71c6-4562-80da-37c58a33ec62 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:01.613466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:01.614054 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.615021 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.615162 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.615803 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:01.617817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:01.644911 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1442/2889] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:23:01 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:01.649425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:01.659400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5a9f6-1601-432b-8d1b-f7fb8844854e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:01.715259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:01.715879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.716165 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.716442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.717709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:01.718972 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:01.742407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1448/2890] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:23:01 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:01.746472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:01.757485 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e87510-2553-4949-b00f-ef4119e50753 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:01.762008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:01.762591 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.762868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.763137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.763772 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:01.765385 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:01.775547 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1443/2891] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:01.779492 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:01.789298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f294f58-6bfd-4d4b-8e60-4a37a19ce465 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:01.793866 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67077d80-7e94-4118-9cd8-bd885b9a22e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:01.794452 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077d80-7e94-4118-9cd8-bd885b9a22e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.794728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077d80-7e94-4118-9cd8-bd885b9a22e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.794999 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077d80-7e94-4118-9cd8-bd885b9a22e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.795639 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67077d80-7e94-4118-9cd8-bd885b9a22e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:01.797253 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67077d80-7e94-4118-9cd8-bd885b9a22e9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:01.808222 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1449/2892] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:01.812191 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:01.822638 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e100d7-e62a-4ce6-98af-bd38c11e17ba admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:01.827169 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:01.827795 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.828103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.828376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] PATH_INFO: `/v3/projects/60765656dd1a4e839a4d3719280b7af5/users/ba003fde113649d99ac83b171ad2035c/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.842969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60765656dd1a4e839a4d3719280b7af5, user_id=ba003fde113649d99ac83b171ad2035c, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:01.845707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:01.848790 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:01.860181 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1444/2893] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:23:01 2022] PUT /identity/v3/projects/60765656dd1a4e839a4d3719280b7af5/users/ba003fde113649d99ac83b171ad2035c/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:01.870829 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc724-c4ff-4da8-a9b1-0535633b63ec None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:01.871114 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc724-c4ff-4da8-a9b1-0535633b63ec None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:01.871413 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc724-c4ff-4da8-a9b1-0535633b63ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:01.899065 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fc6cc724-c4ff-4da8-a9b1-0535633b63ec None None] MFA Rules not processed for user `ba003fde113649d99ac83b171ad2035c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:01.900307 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fc6cc724-c4ff-4da8-a9b1-0535633b63ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:01.963202 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1450/2894] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:01.967648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.029357 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6afd87-d334-4968-be9c-c308fbd685d3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.033975 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.034572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.034855 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.035130 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.035764 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.038226 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.064852 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1445/2895] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:23:01 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:02.069501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cc724-c4ff-4da8-a9b1-0535633b63ec None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.080066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cc724-c4ff-4da8-a9b1-0535633b63ec None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.123928 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:02.137660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.138259 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.138537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.138810 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.139428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.141356 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.163586 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1451/2896] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:23:02 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:02.167739 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.178211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27a54f-39c9-41a3-85b7-6261846c4244 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.182681 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.183264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.183541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.183813 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.184504 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.186111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.197208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1446/2897] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:02.201264 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.211763 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce3958-b633-476c-b9a8-5dd0eb0ce18e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.216344 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.216978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.217258 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.217529 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] PATH_INFO: `/v3/projects/2148e4f7ebda45e8ae21cf7e007660ac/users/1471992d94e34d699df889fab0e3ab13/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.232910 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2148e4f7ebda45e8ae21cf7e007660ac, user_id=1471992d94e34d699df889fab0e3ab13, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.235647 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.249650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1452/2898] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:23:02 2022] PUT /identity/v3/projects/2148e4f7ebda45e8ae21cf7e007660ac/users/1471992d94e34d699df889fab0e3ab13/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:02.253754 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.264242 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4ddda-e215-4f18-9a33-af9bf1b42055 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.321470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6112a19-363e-4b63-9e94-e77f785947fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.322027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6112a19-363e-4b63-9e94-e77f785947fb admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.322309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6112a19-363e-4b63-9e94-e77f785947fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.322583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6112a19-363e-4b63-9e94-e77f785947fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.323216 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6112a19-363e-4b63-9e94-e77f785947fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.324844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6112a19-363e-4b63-9e94-e77f785947fb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.334672 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1447/2899] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:02.338806 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.349081 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1079-5ba2-4585-af50-089cc27c7726 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.353746 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.354396 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.354679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.354955 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] PATH_INFO: `/v3/projects/2148e4f7ebda45e8ae21cf7e007660ac/users/1471992d94e34d699df889fab0e3ab13/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.356940 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2148e4f7ebda45e8ae21cf7e007660ac, user_id=1471992d94e34d699df889fab0e3ab13, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.359583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.370803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1453/2900] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:23:02 2022] PUT /identity/v3/projects/2148e4f7ebda45e8ae21cf7e007660ac/users/1471992d94e34d699df889fab0e3ab13/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:02.382057 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8f5f2-cfdb-4761-bd59-2418663f9a33 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.382342 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8f5f2-cfdb-4761-bd59-2418663f9a33 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.382612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8f5f2-cfdb-4761-bd59-2418663f9a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.412245 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c9c8f5f2-cfdb-4761-bd59-2418663f9a33 None None] MFA Rules not processed for user `1471992d94e34d699df889fab0e3ab13`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:02.413452 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c9c8f5f2-cfdb-4761-bd59-2418663f9a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:02.465285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:02.484759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1448/2901] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:02.497306 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.499154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:02.557669 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474eb99-b332-4a90-beb9-8f256c6078ed admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.561330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.561924 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.562203 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.562471 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.572167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.573523 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.582709 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1454/2902] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:02.654874 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8f5f2-cfdb-4761-bd59-2418663f9a33 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:02.665323 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8f5f2-cfdb-4761-bd59-2418663f9a33 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:02.668957 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f524a-2631-4bad-9961-928c19a7477e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:02.669551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f524a-2631-4bad-9961-928c19a7477e service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:02.669831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f524a-2631-4bad-9961-928c19a7477e service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:02.670103 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f524a-2631-4bad-9961-928c19a7477e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:02.726850 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f524a-2631-4bad-9961-928c19a7477e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:02.728336 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f524a-2631-4bad-9961-928c19a7477e service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:02.737470 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1449/2903] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:03.000609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:03.061987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af3d0-54a3-4a08-a89f-9de144b1260a service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:03.066484 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:03.067101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:03.067378 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:03.067650 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] PATH_INFO: `/v3/users/ba003fde113649d99ac83b171ad2035c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:03.068801 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba003fde113649d99ac83b171ad2035c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:03.070737 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:03.113278 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:03.116436 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba003fde113649d99ac83b171ad2035c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.129167 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba003fde113649d99ac83b171ad2035c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.136425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba003fde113649d99ac83b171ad2035c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.137039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba003fde113649d99ac83b171ad2035c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.143809 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba003fde113649d99ac83b171ad2035c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.150379 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1455/2904] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:23:02 2022] DELETE /identity/v3/users/ba003fde113649d99ac83b171ad2035c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:03.155090 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:03.155642 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:03.155766 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:03.175455 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:03.176684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:03.234653 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1450/2905] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:03.244400 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:03.245803 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:03.303189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472c54-814b-423e-bb49-eff65ecac866 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:03.306892 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:03.307490 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:03.307772 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:03.308076 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:03.317005 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:03.318588 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:03.328012 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1456/2906] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:03.593033 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:03.594449 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:03.607920 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e709e5-fb9b-414e-8098-445a6b69b0e4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:03.612762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:03.613388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:03.613717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:03.613996 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] PATH_INFO: `/v3/projects/60765656dd1a4e839a4d3719280b7af5` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:03.615457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=60765656dd1a4e839a4d3719280b7af5)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:03.617431 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:03.667173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60765656dd1a4e839a4d3719280b7af5', 'request_id': 'req-6e8fb373-f133-442d-bda7-8bdd2482e50e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.672651 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60765656dd1a4e839a4d3719280b7af5', 'request_id': 'req-6e8fb373-f133-442d-bda7-8bdd2482e50e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.679498 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60765656dd1a4e839a4d3719280b7af5', 'request_id': 'req-6e8fb373-f133-442d-bda7-8bdd2482e50e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.691918 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] The token cache is being invalidate because project 60765656dd1a4e839a4d3719280b7af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:23:03.692208 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.693424 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.694919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1451/2907] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:23:03 2022] DELETE /identity/v3/projects/60765656dd1a4e839a4d3719280b7af5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:03.698934 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:03.709197 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b081ef-9a2e-44a0-a51a-038b6c299390 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:03.766298 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:03.766926 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:03.767208 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:03.767482 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] PATH_INFO: `/v3/users/1471992d94e34d699df889fab0e3ab13` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:03.768630 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1471992d94e34d699df889fab0e3ab13)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:03.770590 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:03.817447 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1471992d94e34d699df889fab0e3ab13'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.829564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1471992d94e34d699df889fab0e3ab13'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.831147 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:03.837634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1471992d94e34d699df889fab0e3ab13'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.838428 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1471992d94e34d699df889fab0e3ab13'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.846154 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f48bb1d3-1aac-4911-af9e-8bd5b411da71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1471992d94e34d699df889fab0e3ab13'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:03.851978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1457/2908] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:23:03 2022] DELETE /identity/v3/users/1471992d94e34d699df889fab0e3ab13 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:04.120744 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.122172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:04.136846 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fb373-f133-442d-bda7-8bdd2482e50e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.146321 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:04.196334 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.197069 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.197229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.197503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] PATH_INFO: `/v3/projects/2148e4f7ebda45e8ae21cf7e007660ac` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.198614 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2148e4f7ebda45e8ae21cf7e007660ac)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.200551 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.246785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2148e4f7ebda45e8ae21cf7e007660ac', 'request_id': 'req-d529c219-9da4-4d99-b61d-ec158d264535'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:04.252168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2148e4f7ebda45e8ae21cf7e007660ac', 'request_id': 'req-d529c219-9da4-4d99-b61d-ec158d264535'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:04.258581 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2148e4f7ebda45e8ae21cf7e007660ac', 'request_id': 'req-d529c219-9da4-4d99-b61d-ec158d264535'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:04.270684 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] The token cache is being invalidate because project 2148e4f7ebda45e8ae21cf7e007660ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:23:04.271243 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:04.272168 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d529c219-9da4-4d99-b61d-ec158d264535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:04.273641 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1452/2909] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:23:04 2022] DELETE /identity/v3/projects/2148e4f7ebda45e8ae21cf7e007660ac => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:04.278498 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295c71-bc62-429a-a065-a7dac0ce0893 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.278816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295c71-bc62-429a-a065-a7dac0ce0893 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.279094 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295c71-bc62-429a-a065-a7dac0ce0893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.298766 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0e295c71-bc62-429a-a065-a7dac0ce0893 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:04.299987 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e295c71-bc62-429a-a065-a7dac0ce0893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:04.358743 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1458/2910] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 11:23:04.368539 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd268c-263a-4bd5-8f5e-16768cfc8fba None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.368831 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd268c-263a-4bd5-8f5e-16768cfc8fba None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.369109 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd268c-263a-4bd5-8f5e-16768cfc8fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.389241 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0bdd268c-263a-4bd5-8f5e-16768cfc8fba None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:04.390461 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0bdd268c-263a-4bd5-8f5e-16768cfc8fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:04.399552 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1453/2911] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:04.403824 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295c71-bc62-429a-a065-a7dac0ce0893 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.414014 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295c71-bc62-429a-a065-a7dac0ce0893 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.418534 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.419118 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.419392 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.419659 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.420341 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:23:04.420638 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.422211 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.434173 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1459/2912] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:23:04.438471 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd268c-263a-4bd5-8f5e-16768cfc8fba None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.448059 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd268c-263a-4bd5-8f5e-16768cfc8fba None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.452648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.453230 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.453537 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.453811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.454432 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.456359 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.483983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1454/2913] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:23:04 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:04.488227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.498631 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04602fad-cf16-41b7-afcf-1f8b5cd73eb4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.554728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.555328 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.555614 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.555918 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.556541 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.558455 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.578855 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1460/2914] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:23:04 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:04.582945 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.594009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e27c4-743c-47bd-9f49-28ef52e3d26f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.599383 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.600000 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.600272 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.600540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.601172 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.602769 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.612658 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1455/2915] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:04.616530 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.626214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517430b-f8a5-42d1-9060-fdc00beb569d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.630818 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175c101-5da8-4535-b6a7-01ebea76ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.631402 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c101-5da8-4535-b6a7-01ebea76ab8a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.631679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c101-5da8-4535-b6a7-01ebea76ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.631984 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c101-5da8-4535-b6a7-01ebea76ab8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.632619 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175c101-5da8-4535-b6a7-01ebea76ab8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.634255 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175c101-5da8-4535-b6a7-01ebea76ab8a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.644627 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1461/2916] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:04.649289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.659086 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3921ff-1343-4a43-bf4b-add246a73ae7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.664367 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.665050 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.665331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.665650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] PATH_INFO: `/v3/projects/a3890b72a80c4341a806711ced8a39ce/users/8af6a1061e8c4aab9536fcda3a69d8b3/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.680556 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3890b72a80c4341a806711ced8a39ce, user_id=8af6a1061e8c4aab9536fcda3a69d8b3, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.683331 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.699386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1456/2917] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:23:04 2022] PUT /identity/v3/projects/a3890b72a80c4341a806711ced8a39ce/users/8af6a1061e8c4aab9536fcda3a69d8b3/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:04.710123 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be697-51ef-4ce2-b30c-75d6553c6019 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.710617 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be697-51ef-4ce2-b30c-75d6553c6019 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.710736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be697-51ef-4ce2-b30c-75d6553c6019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.738501 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-431be697-51ef-4ce2-b30c-75d6553c6019 None None] MFA Rules not processed for user `8af6a1061e8c4aab9536fcda3a69d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:04.739773 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-431be697-51ef-4ce2-b30c-75d6553c6019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:04.778006 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:04.805421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1462/2918] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:04.809997 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.852926 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:04.873502 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5a75f-94ad-4aaa-92f6-bb8fa86ceecf admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.878167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.878759 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.879042 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.879316 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.880078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.882107 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:04.908426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1457/2919] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:23:04 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:04.913066 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431be697-51ef-4ce2-b30c-75d6553c6019 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:04.923685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431be697-51ef-4ce2-b30c-75d6553c6019 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:04.980017 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:04.980645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:04.980922 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:04.981194 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:04.981823 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:04.983718 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:05.005753 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1463/2920] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:23:04 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:05.010315 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:05.021303 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86111fc8-f9e8-472f-a3c5-5d23ef46a397 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:05.026134 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:05.026713 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:05.026991 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:05.027265 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:05.027927 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:05.029514 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:05.040968 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1458/2921] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 11:23:05.045039 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:05.054914 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5fb8d-6cd2-41b4-8c2c-4c6d0f430a5f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:05.059609 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:05.060497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:05.060836 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:05.061110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] PATH_INFO: `/v3/projects/65b4ed947f2142089a5737584eabafc9/users/49c921805af54220b65b6816032dad49/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:05.075963 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b4ed947f2142089a5737584eabafc9, user_id=49c921805af54220b65b6816032dad49, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:05.078587 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:05.091880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1464/2922] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:23:05 2022] PUT /identity/v3/projects/65b4ed947f2142089a5737584eabafc9/users/49c921805af54220b65b6816032dad49/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:05.095743 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:05.105495 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c3e34-cbf7-4ea4-9fc0-83e6c4cb0855 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:05.162250 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d984b0-f448-496c-a5bf-85bce3597056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:05.162844 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d984b0-f448-496c-a5bf-85bce3597056 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:05.163122 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d984b0-f448-496c-a5bf-85bce3597056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:05.163396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d984b0-f448-496c-a5bf-85bce3597056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:05.164061 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d984b0-f448-496c-a5bf-85bce3597056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:05.165640 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d984b0-f448-496c-a5bf-85bce3597056 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:05.175222 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1459/2923] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:05.179950 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:05.190214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f03106-0a46-4a61-9e31-027db69abdb8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:05.195564 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:05.196237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:05.196514 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:05.196785 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] PATH_INFO: `/v3/projects/65b4ed947f2142089a5737584eabafc9/users/49c921805af54220b65b6816032dad49/roles/4c41d809cc274ff5871dc8092f05715a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:05.198732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b4ed947f2142089a5737584eabafc9, user_id=49c921805af54220b65b6816032dad49, role_id=4c41d809cc274ff5871dc8092f05715a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:05.201468 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:05.212108 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1465/2924] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:23:05 2022] PUT /identity/v3/projects/65b4ed947f2142089a5737584eabafc9/users/49c921805af54220b65b6816032dad49/roles/4c41d809cc274ff5871dc8092f05715a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:05.224269 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14f37b-23e4-4ced-9147-15d9ce33ef92 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:05.224849 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14f37b-23e4-4ced-9147-15d9ce33ef92 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:05.225096 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14f37b-23e4-4ced-9147-15d9ce33ef92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:05.252546 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cb14f37b-23e4-4ced-9147-15d9ce33ef92 None None] MFA Rules not processed for user `49c921805af54220b65b6816032dad49`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:05.253662 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cb14f37b-23e4-4ced-9147-15d9ce33ef92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:05.322047 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1460/2925] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:05.332634 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:05.334858 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:05.392090 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d292e56-faef-484c-b5b3-79f84b254b92 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:05.396190 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218cda5-3484-455a-9a1d-6b5a6c504bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:05.397101 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218cda5-3484-455a-9a1d-6b5a6c504bde service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:05.397374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218cda5-3484-455a-9a1d-6b5a6c504bde service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:05.397643 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218cda5-3484-455a-9a1d-6b5a6c504bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:05.442547 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:05.456414 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218cda5-3484-455a-9a1d-6b5a6c504bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:05.457876 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218cda5-3484-455a-9a1d-6b5a6c504bde service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:05.467364 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1466/2926] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:07.999426 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14f37b-23e4-4ced-9147-15d9ce33ef92 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:08.038052 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:08.062733 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14f37b-23e4-4ced-9147-15d9ce33ef92 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:08.067176 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:08.067817 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:08.068127 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:08.068400 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] PATH_INFO: `/v3/users/8af6a1061e8c4aab9536fcda3a69d8b3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:08.069528 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af6a1061e8c4aab9536fcda3a69d8b3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:08.071450 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:08.120595 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af6a1061e8c4aab9536fcda3a69d8b3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.132883 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af6a1061e8c4aab9536fcda3a69d8b3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.133585 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af6a1061e8c4aab9536fcda3a69d8b3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.141046 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af6a1061e8c4aab9536fcda3a69d8b3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.141748 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af6a1061e8c4aab9536fcda3a69d8b3'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.155227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1461/2927] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:23:07 2022] DELETE /identity/v3/users/8af6a1061e8c4aab9536fcda3a69d8b3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:08.160646 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:08.161834 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:08.161957 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:08.182540 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:08.183736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:08.242227 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1467/2928] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:08.251856 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:08.253229 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:08.332035 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6611d-2f2f-4a71-9468-328b57a84a8f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:08.336353 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:08.336947 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:08.337227 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:08.337503 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:08.346284 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:08.347706 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:08.357472 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1462/2929] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:08.622355 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:08.623793 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:08.636442 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f04b43-8c08-4534-9dfd-80cab688c0c1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:08.641086 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:08.641726 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:08.642000 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:08.642271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] PATH_INFO: `/v3/projects/a3890b72a80c4341a806711ced8a39ce` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:08.643388 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3890b72a80c4341a806711ced8a39ce)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:08.645330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:08.693226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3890b72a80c4341a806711ced8a39ce', 'request_id': 'req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.698664 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3890b72a80c4341a806711ced8a39ce', 'request_id': 'req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.704904 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3890b72a80c4341a806711ced8a39ce', 'request_id': 'req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.717645 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] The token cache is being invalidate because project a3890b72a80c4341a806711ced8a39ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:23:08.717931 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.718546 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.720107 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1468/2930] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:23:08 2022] DELETE /identity/v3/projects/a3890b72a80c4341a806711ced8a39ce => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:08.724075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:08.733703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274fb5c-7246-4ed9-80e2-00a11d480083 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:08.766724 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:08.793027 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:08.793655 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:08.793936 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:08.794211 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] PATH_INFO: `/v3/users/49c921805af54220b65b6816032dad49` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:08.795341 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c921805af54220b65b6816032dad49)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:08.797339 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:08.842648 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c921805af54220b65b6816032dad49'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.854785 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c921805af54220b65b6816032dad49'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.855440 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c921805af54220b65b6816032dad49'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.862414 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c921805af54220b65b6816032dad49'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.863034 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-01c93a45-5dad-45b8-b50f-7a0e9a62660b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c921805af54220b65b6816032dad49'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:08.875287 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1463/2931] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:23:08 2022] DELETE /identity/v3/users/49c921805af54220b65b6816032dad49 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:09.152788 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.154266 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:09.174322 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9969f-1eaa-4ed0-9690-4046aa17ea7c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.232237 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.232869 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.233149 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.233425 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] PATH_INFO: `/v3/projects/65b4ed947f2142089a5737584eabafc9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.234542 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b4ed947f2142089a5737584eabafc9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.236497 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.247241 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:09.284279 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b4ed947f2142089a5737584eabafc9', 'request_id': 'req-b53a7ce3-f5ce-442b-bbb9-338702fc147c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:09.289849 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b4ed947f2142089a5737584eabafc9', 'request_id': 'req-b53a7ce3-f5ce-442b-bbb9-338702fc147c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:09.295879 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b4ed947f2142089a5737584eabafc9', 'request_id': 'req-b53a7ce3-f5ce-442b-bbb9-338702fc147c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:09.308887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] The token cache is being invalidate because project 65b4ed947f2142089a5737584eabafc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:23:09.309180 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:09.309794 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b53a7ce3-f5ce-442b-bbb9-338702fc147c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:09.311271 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1469/2932] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:23:09 2022] DELETE /identity/v3/projects/65b4ed947f2142089a5737584eabafc9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:09.316151 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6def9e-10f8-4c22-90f5-1804b3d2bf5f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.316666 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6def9e-10f8-4c22-90f5-1804b3d2bf5f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.316762 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6def9e-10f8-4c22-90f5-1804b3d2bf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.336811 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-eb6def9e-10f8-4c22-90f5-1804b3d2bf5f None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:09.338008 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eb6def9e-10f8-4c22-90f5-1804b3d2bf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:09.401667 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1464/2933] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:09.409407 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19af63-5bee-4e8a-9ec3-2169134d0c33 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.409694 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19af63-5bee-4e8a-9ec3-2169134d0c33 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.409974 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19af63-5bee-4e8a-9ec3-2169134d0c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.431463 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cd19af63-5bee-4e8a-9ec3-2169134d0c33 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:09.432702 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cd19af63-5bee-4e8a-9ec3-2169134d0c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:09.440502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1470/2934] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:09.445193 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6def9e-10f8-4c22-90f5-1804b3d2bf5f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.454717 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6def9e-10f8-4c22-90f5-1804b3d2bf5f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.459857 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.460453 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.460735 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.461009 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.461649 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 11:23:09.461951 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.463541 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.476112 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1465/2935] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Oct 6 11:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 11:23:09.480168 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19af63-5bee-4e8a-9ec3-2169134d0c33 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.489138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19af63-5bee-4e8a-9ec3-2169134d0c33 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.494799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.495380 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.495658 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.495964 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.496594 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.498466 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.525605 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1471/2936] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Oct 6 11:23:09 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:09.530019 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.540937 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14e82e-9c93-4851-954b-9a2cc93c71b7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.584037 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:09.602578 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.603178 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.603454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.603721 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.604499 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.607412 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.629309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1466/2937] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Oct 6 11:23:09 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 11:23:09.633720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.643728 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37beea-76e9-494b-9c40-a87962e4981c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.648583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.649172 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.649445 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.649711 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.650362 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.651970 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.662846 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1472/2938] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:09.666705 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.677599 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2978b-a969-4715-9c04-cb914fc63399 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.682285 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c6a60-ff8e-41bb-90be-43d921e41b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.682868 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c6a60-ff8e-41bb-90be-43d921e41b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.683144 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c6a60-ff8e-41bb-90be-43d921e41b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.683411 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c6a60-ff8e-41bb-90be-43d921e41b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.684077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c6a60-ff8e-41bb-90be-43d921e41b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.685650 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c6a60-ff8e-41bb-90be-43d921e41b33 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.696261 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1467/2939] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Oct 6 11:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 11:23:09.700429 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.710434 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a271583-b8d0-4e39-a4bb-818f88355d45 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.715052 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.715689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.716010 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.716289 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] PATH_INFO: `/v3/projects/cc900882deba46cab54226197527a38b/users/ddcadd9fa2144ae5a3177966c6200767/roles/1a3c6cbe3fb44e9a812652129507717e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.731186 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc900882deba46cab54226197527a38b, user_id=ddcadd9fa2144ae5a3177966c6200767, role_id=1a3c6cbe3fb44e9a812652129507717e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.733890 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.748421 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1473/2940] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Oct 6 11:23:09 2022] PUT /identity/v3/projects/cc900882deba46cab54226197527a38b/users/ddcadd9fa2144ae5a3177966c6200767/roles/1a3c6cbe3fb44e9a812652129507717e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:09.758692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e621518-2b80-42a8-a540-f7cead65b3f8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.758983 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e621518-2b80-42a8-a540-f7cead65b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.759264 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e621518-2b80-42a8-a540-f7cead65b3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.788306 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3e621518-2b80-42a8-a540-f7cead65b3f8 None None] MFA Rules not processed for user `ddcadd9fa2144ae5a3177966c6200767`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:09.789521 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3e621518-2b80-42a8-a540-f7cead65b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:09.852466 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1468/2941] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:09.863510 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:09.865943 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:09.923201 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c524da-43cb-49c6-a5bf-e982dc06dfb8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:09.927389 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf3a3a-fc11-4350-a44b-5ca92a3b0e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ecccd27dfa04cf985c9fd632ce2fe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:09.928138 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf3a3a-fc11-4350-a44b-5ca92a3b0e46 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:09.928417 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf3a3a-fc11-4350-a44b-5ca92a3b0e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:09.928685 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf3a3a-fc11-4350-a44b-5ca92a3b0e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:09.936799 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf3a3a-fc11-4350-a44b-5ca92a3b0e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:09.938214 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf3a3a-fc11-4350-a44b-5ca92a3b0e46 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:09.947550 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1474/2942] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:19.848977 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:19.943967 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:23:55.560559 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e621518-2b80-42a8-a540-f7cead65b3f8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:55.625583 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e621518-2b80-42a8-a540-f7cead65b3f8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:55.630080 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:55.630707 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:55.630986 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:55.631258 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] PATH_INFO: `/v3/users/ddcadd9fa2144ae5a3177966c6200767` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:55.632396 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddcadd9fa2144ae5a3177966c6200767)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:55.634319 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:55.680786 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddcadd9fa2144ae5a3177966c6200767'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:55.692388 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddcadd9fa2144ae5a3177966c6200767'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:55.693056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddcadd9fa2144ae5a3177966c6200767'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:55.700361 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddcadd9fa2144ae5a3177966c6200767'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:55.701075 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddcadd9fa2144ae5a3177966c6200767'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:55.714726 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1469/2943] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:23:55 2022] DELETE /identity/v3/users/ddcadd9fa2144ae5a3177966c6200767 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:23:55.719263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:55.719797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:55.719797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:55.739461 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] MFA Rules not processed for user `dd261f057dda47fbbc28e2f5e9b2f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 11:23:55.740689 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:55.798720 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1475/2944] 10.222.0.52 () {58 vars in 958 bytes} [Thu Oct 6 11:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 11:23:55.808389 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:55.809876 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:55.868959 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1761982-9336-4cad-a991-91004e017956 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:55.872685 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:55.873289 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:55.873567 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:55.873839 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:55.882692 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:55.884182 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:55.894137 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1470/2945] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:23:56.159693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:23:56.161295 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:23:56.174559 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cfbb9-3318-45a1-a73d-5ab68816b529 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:23:56.179188 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:23:56.179816 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:23:56.180120 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:23:56.180393 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] PATH_INFO: `/v3/projects/cc900882deba46cab54226197527a38b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:23:56.181504 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc900882deba46cab54226197527a38b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:23:56.184480 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:23:56.231860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc900882deba46cab54226197527a38b', 'request_id': 'req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:56.237475 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc900882deba46cab54226197527a38b', 'request_id': 'req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:56.243697 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc900882deba46cab54226197527a38b', 'request_id': 'req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:56.256175 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] The token cache is being invalidate because project cc900882deba46cab54226197527a38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:23:56.256467 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:56.257078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:23:56.258528 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1476/2946] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:23:56 2022] DELETE /identity/v3/projects/cc900882deba46cab54226197527a38b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:24:05.889969 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:24:06.253178 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:24:07.698974 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:07.700438 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:07.712077 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284acbc-c5a7-4f45-9f7a-7c200f99c37e service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:07.762906 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:07.763520 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:07.763798 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:07.764101 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:07.765848 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:07.857919 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:07.859376 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:07.880699 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1471/2947] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:24:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:24:09.065040 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:09.066474 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:09.078502 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1b0ae-564f-4fbe-bcaf-9dc27d1dc795 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:09.082008 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:09.082601 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:09.082880 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:09.083153 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:09.084860 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:09.102822 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:09.104959 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:09.119941 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1477/2948] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:24:09.994600 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:09.996840 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:10.011117 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aac05-75ee-4ecd-b52b-7ae0d69d9e98 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:10.016921 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91413e6647284487bae08bb7eb02da0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:10.017612 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:10.017932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:10.018309 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:10.020333 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:10.033948 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:10.035770 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:10.047317 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1472/2949] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:24:19.114527 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:24:19.957478 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:19.958896 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:19.969736 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc0dce-38ac-40c1-9369-cd845345447e service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:19.973444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5c892640b3e409b9a99b4ca1d34a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:19.973978 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:19.974261 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:19.974537 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:19.976212 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:20.039797 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:20.041311 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:20.044519 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=72 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:24:20.051740 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1478/2950] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:24:23.151183 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:23.152716 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:23.228928 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9e50e-e373-4fd4-a916-f499b5c12509 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:23.234579 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:23.235268 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:23.235483 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:23.235750 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:23.237538 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:23.255869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:23.255869 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:23.266543 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1473/2951] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:24:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:24:26.329163 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:26.330732 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:26.343196 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158cce2-8f4c-4e37-9e68-1b5873f324d0 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:26.348078 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:26.348657 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:26.348965 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:26.349238 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] PATH_INFO: `/v3/users/acf06fe641e94b5b91145ed230ac7213` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:26.350374 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf06fe641e94b5b91145ed230ac7213)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:26.352321 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:26.403863 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf06fe641e94b5b91145ed230ac7213'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.415660 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf06fe641e94b5b91145ed230ac7213'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.423083 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf06fe641e94b5b91145ed230ac7213'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.423684 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf06fe641e94b5b91145ed230ac7213'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.430632 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf06fe641e94b5b91145ed230ac7213'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.437177 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1479/2952] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:24:26 2022] DELETE /identity/v3/users/acf06fe641e94b5b91145ed230ac7213 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:24:26.611999 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:26.624173 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35336b93-5ecc-4116-982c-bc237290600a service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:26.681290 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:26.681910 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:26.682187 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:26.682459 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] PATH_INFO: `/v3/projects/0934343645484d01b9b48a40bbf0b671` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:26.683560 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0934343645484d01b9b48a40bbf0b671)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:26.686111 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:26.732386 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0934343645484d01b9b48a40bbf0b671', 'request_id': 'req-d0490d31-2dca-4625-aeb0-19d9a311c4ec'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.737727 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0934343645484d01b9b48a40bbf0b671', 'request_id': 'req-d0490d31-2dca-4625-aeb0-19d9a311c4ec'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.744159 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0934343645484d01b9b48a40bbf0b671', 'request_id': 'req-d0490d31-2dca-4625-aeb0-19d9a311c4ec'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.756771 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] The token cache is being invalidate because project 0934343645484d01b9b48a40bbf0b671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:24:26.757056 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.761657 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.763454 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1474/2953] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:24:26 2022] DELETE /identity/v3/projects/0934343645484d01b9b48a40bbf0b671 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:24:26.767990 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:26.769641 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:26.783110 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bc432-8ee5-4005-b9f8-d6b96725f59c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:26.837784 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:26.838403 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:26.838679 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:26.838951 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] PATH_INFO: `/v3/users/7ae698b7b22247c79137ae6157aeae70` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:26.840606 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae698b7b22247c79137ae6157aeae70)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:26.842499 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:26.848016 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:24:26.889469 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae698b7b22247c79137ae6157aeae70'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.901300 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae698b7b22247c79137ae6157aeae70'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.909226 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae698b7b22247c79137ae6157aeae70'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.909852 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae698b7b22247c79137ae6157aeae70'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.916774 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae698b7b22247c79137ae6157aeae70'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:26.923171 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1480/2954] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Oct 6 11:24:26 2022] DELETE /identity/v3/users/7ae698b7b22247c79137ae6157aeae70 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 11:24:27.092703 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:24:27.094167 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:24:27.108905 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0490d31-2dca-4625-aeb0-19d9a311c4ec admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:24:27.127240 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:24:27.176104 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd261f057dda47fbbc28e2f5e9b2f3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46752b92e78642e4b79f5ef50f2e51da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:24:27.176745 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:24:27.177026 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:24:27.177307 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] PATH_INFO: `/v3/projects/34ac15ce8cd2461da24ef95a72683234` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:24:27.178457 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ac15ce8cd2461da24ef95a72683234)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:24:27.180419 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:24:27.227224 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ac15ce8cd2461da24ef95a72683234', 'request_id': 'req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:27.233572 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ac15ce8cd2461da24ef95a72683234', 'request_id': 'req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:27.240078 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ac15ce8cd2461da24ef95a72683234', 'request_id': 'req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:27.252540 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] The token cache is being invalidate because project 34ac15ce8cd2461da24ef95a72683234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 11:24:27.252828 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:27.253451 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7e44ae4c-321e-4cb9-98ab-a54ea6d31f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 11:24:27.254932 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1475/2955] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Oct 6 11:24:27 2022] DELETE /identity/v3/projects/34ac15ce8cd2461da24ef95a72683234 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 11:24:36.920330 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:24:37.249617 dv-cinder-860209-2-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 11:27:18.807158 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 11:27:18.808560 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:27:18.877583 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf42e6c-7a69-4b25-b73a-9afe1bbb0986 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 11:27:18.937285 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23718ed9-436a-4c9c-9044-3d48ba317b44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49cbfb2f4ddb42c0b299311615c7b0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0751f4839c4258b9a815a044d9c388', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 11:27:18.937887 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23718ed9-436a-4c9c-9044-3d48ba317b44 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 11:27:18.938166 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23718ed9-436a-4c9c-9044-3d48ba317b44 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 11:27:18.938444 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23718ed9-436a-4c9c-9044-3d48ba317b44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 11:27:18.940189 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23718ed9-436a-4c9c-9044-3d48ba317b44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 11:27:18.951263 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23718ed9-436a-4c9c-9044-3d48ba317b44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 11:27:18.952857 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23718ed9-436a-4c9c-9044-3d48ba317b44 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 11:27:18.963059 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1481/2956] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Oct 6 11:27:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 11:27:28.958693 dv-cinder-860209-2-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=58 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}